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FOBEWORD 



When the U.S. Office of Education 
was chartered m 1867, one charge to its 
commissioners was to determine the nation's 
progress in education. The National 
Assessment of Educational Progress (^NAEP) 
was initiated a century later to address, in a 
systematic way, that charge. 

Since 1969, the National Assessment 
has gathered information about levels of 
educational achievement across the country 
and reported its findings to the nation, tt has 
surveyed the attainments of 9-year-olds, 
13-yeaf-olds, 17-year-olds and adults in 
art, career and occupational development, 
citizenship, literature, mathematics, music, 
reading, science, social studies and writing. All 
areas have been periodically reassessed in 
order to detect any important changes. To 
date. National Assessment has interviewed 
and tested nearly 1,000,000 young Americans. 

Learning-area assessments evolve from a 
consensus process. Each assessment is the 
product of several years of work by a great 
many educators, scholars and lay persons 
from all over the nation Initially, these people 
design objectives for each subject area, 
proposing general goals they feel Americans 
should be achieving in the course of their 
education. After careful review, these 
objectives are given to writers, whose task is 
to create exercises (items) appropriate to trte 
objectives. 



When the exercises have passed 
extensive reviews by subject-area specialists, 
measurement experts and lay persons, they 
are administereddo probability samples. The 
people in these samples are selected in such a 
way that the results of their assessment can 
be generalized to an entire national 
population. That is. on the basis of the 
performance of about 2.500 9-year-olds on a 
given exercise, we can make generalizations 
about the probable performance of all 9-year- 
olds in the nation. 

After assessment data have been 
collected, scored and analyzed, the National 
Assessment publishes reports and 
disseminates the results as widely as possible. 
Not all exercises are released for publication 
Because NAEP will readminister some of the 
same exercises in the future to determine 
whether the performance levels of Americans 
have increased, remained stable or 
decreased, it is essential that they not be 
released in order to preserve the integrity of 
the study. 
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Wliy mis Title? 



This report on students' academic 
performance looks beyond the boundaries 
traditionally ascribed to subject areas. It rests 
upon the assumption that in order to 
'inderstand how well people read, we must 
look at their ability to read a range of 
materials and to express and explain their 
interpretations of what they have read. 
Accordingly, the national assessment of 
reading and literature employed a variety of 
techniques to examine students' ability to 
comprehend what they have read, including 
some tasks that asked for relatively extended 
discussion of text material. Th^ model on 
which the report is based implies that initial 
comprehension of a passage can be expanded 
and refined through reflection and 
interpretation, and that this in turn leads to a 
bette*- understanding of the material itself. 

Besides giving us the opportunity to 
discuss reading and literature in the larger 
context of literacy— thinking, responding and 
writing — this integrated perspective provides 
the opportunity to contrast students' 
performance bn a range of multiple-choice and 
open-ended tasks, as well as to examine the 
extent to whicr students' performance is 
influenced by tie characteristics of texts they 
are reading. Mi ich of the material on which 
items in this report are based is literary in 
nature, though ihe passages are drawn from a 
vanety of genres, including fictional as well as 
nonfictional textk. We believe that the reading, 
thinking and wn ing skills drawn upon in these 
exercises are edually important in all subject- 
area reading tasks. The literary selections 
chosen for examination here allow us to 
assess relatively \:omplex interpretive and 
analytic skills withbut requinng the specialized 
knowledge and vocabulary of particular 
subject- area fields. \ 



When reading is divorced from the 
process of discussing the meaning of a work 
(as it often is in teaching and testing), 
comprehension can be misunderstood to be a 
sudden "click" of meaning measurable only 
through short-answer and multiple-choice 
questions that require little struggle for full 
understanding. If tasks that require students 
to explain and substantiate their judgments 
and interpretations are relegated solely to 
literature classes rather than to the wide 
variety of situations in which students must 
construct meaning from texts, then the 
complex skills involved in such tasks can 
mistakenly be excluded from reading 
instruction. In fact. rMdlng as traditionally 
assessed through objective tests of 
"comprehension" and responding as 
traditionally measured through open-ended 
writing tasks are aspects of the same 
phenomenon of human understanding. They 
are aspects of learning that work more 
productively in tandem than either does 
separately. 

Many of the specialists who designed the 
reading/literature assessment believe it is time 
for these aspects of reading to be put in 
relation to each other to create a more detailed 
and complete picture of what students are and 
are not learning to do in their reading and 
literature classes. This report represents an 
attempt to move in that direction. 
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Xlia Data Basa 

The National Assessment of Educational 
Progress first assessed reading and literature 
achievement in separate assessments during 
the 1970-71 school year. Since that time, 
reading has been assessed twice as a discrete 
learning area (1974-75 and 1979-80) and 
reading and literature have been combined 
for a joint assessment in 1979-80. Each 
assessment surveyed the achievement and 
attitudes of American 9-, 13- and 17-year-olds, 
using a deaply stratified, multistage probability 
sample design. 

To rne^ure changes in reading 
perfof . nance between 1970-71, 1974-75 and 
1979 80, approximately half of the exercises 
ass'3ssed in the first assessment were 
reassessed in the second and third under 
almost identical administrative conditions. 
To measure the status of reading/literature 
achievement in 1979-80, National Assessment 
consultants developed new objectives and 
developed additional exercises to provide 
coverage of the new objectives. Some 1970-71 
literature items were also reassessed. 

Approximately 29,000 9- year-olds, 41,000 
13-year-olds and 36.000 17-year-olds 
participated in the 1979-80 reading 
assessment. Because National Assessment 
reports results for groups of students, not 
individuals, it is not necessary for each student 
to respond to every item (exercise)/ 

Each respondent completed only one item 
booklet of about 45 minutes in length. Between 
2,500 and 2,900 students responded to each 
booklet. In 1979-80 there were 11 exercise 
booklets for 9- year-olds. 15 booklets for 13- 
year-olds and 14 booklets for 17-y€ar-olds. 

The exercises for each assessment 
were administered by a professional data 
collection staff to minimize the burden on 
participating schools and to maximize 
uniformity of assessment conditions. 
Instructions were recorded on a paced audio 



'National Atttttmant utat tha farm "axarcita" to 
maan an aaaaaamant Itam. Tha tarma "axarclaa" and 
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tape and played back to students to ensure 
that all students moved through the packages 
at the same speed. 

Multiple-choice items were scored by an 
optical scanning machine; open-ended items 
were hand-scored by trained scorers using 
scoring guides developed to define categories 
of acceptable and unacceptable responses. 

National Assessment reports estimated 
percentages of correct responses for single 
items When a report indicates that "85% of the 
17-year-olds gave a correct response," it 
means that an estimated 85% of the 17-year- 
olds would have given the correct response if 
all the 17-year-olds in schools across the 
country had been assessed. In addii jn to 
reporting national results. National 
Assessment provides data on the performance 
of various population subgroups within the 
national population, defined by sex. race, 
region of the country, size and type of 
community lived in and level of parental 
education. National Assessment also 
aggregates percentages of success on various 
sets of items to provide data on changes in 
performance between assessments and on the 
differential performance of population 
subgroups. 

Definitions of the reporting groups follow: 
Region 

The country has been divided into four 
regions: Northeast, Southeast, Central and 
West. States included in each region are 
shown on the following map 




/I 



Sex 

Results are reported for males and 
females. 

Race/Ethnicity 

Results are presented for blacks, whites 
and, in 1979-80, only, Hispanos. 

Level of Parental Education 

National Assessment defines three 
categories of parental-education levels, based 
on students' reports. These categories are: (1) 
those whose parents did not graduate from 
high school, (2) those who have at least one 
parent who graduated from high school and (3) 
those who have at least one parent who has 
had some post- high- school education. 

Typo of Community 

Three extreme community types of special 
interest ar'e defined by an occupational profile 
of the area served by a school as well as by 
the size of the community in which the school 
is located. This is the only reporting category 
that excludes a large number of respondents. 
About two-thirds do not fall into the 
classifications listed below. Results for the 
remaining two-thirds are not reported since 
their performance was similar to that of 
the nation. 

Advantaged- urban (high- metro) 
communitiea. Students in this group attend 
schools in or around cities having a population 
greater than 200,000 where a high proportion 
of the residents are in professional or 
managerial positions. 

Ditadvantaged-urban (low-metro) 
communities. Students in this group attend 
schools in or around cities having a population 
greater than 200,000 where a high proportion 
of the residents are on welfare or are not 
regularly employed. 



Rural communities. Students in this group 
attend schools in areas with a population 
under 10,000 where many of the residents are 
farmers or farm workers. 

Size of Community 

Big cities. Students in this group attend 
schools within the city limits of cities having a 
1970 census population over 200,000. 

Fringes around big cities. Students in this 
group attend schools within metropolitan areas 
(1970 U.S. Bureau of the Census urbanized 
areas) served by cities having a population 
greater than 200,000 but outside the city limits. 

Medium cities. Students in this group 
attend schools in cities having a population 
between 25.000 and 200,000 not classified in. 
the fringes-around-big-cities category. 

Small places. Studer*s in this group 
attend schools in communities having a 
population less than 25,000, not classified in 
the fringes-around-big-cities category. 

Scoring 

Scoring and computer recording of data 
were contracted to Westinghouse DataScore 
Systems (formerly the Measurement Research 
Center), Iowa City, Iowa, for all three reading 
and both literature assessments. 

In the 1979-80 assessment, more than 
90% of the items were multiple-choice and the 
rest were open-ended. Responses to 
multiple-choice exercises were read directly 
by an optical scanning machine. The scoring 
contractor employed a special staff to hand 
score open-ended exercises. Scorers were 
responsible for categorizing open-ended 
responses, using sconng guides that defined 
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categories of acceptable and unacceptable 
responses. They tnen coded this inform atior; 
into ovals that could be read by the optical 
scanning machine. 

For changes m performance to be 
measured accurately, scoring ^had to be 
the same for responses collected in each 
assessment year For multiple-choice itt-r^s, 
the same responses were scored corre^i . i 
each year Some open-ended items were 
short-answer reading items requiring objective 
scoring of a clerical nature. These were all 
reasr^ssed items, and identical guides were 
used in 1979-80 as were used to categorize 
the 1970-71 and 1974-75 responses. Scorers 
were trained using sample responses from all 
three assessment years. Quality-control 
procedures were conducted by having scorers 
rescore papers from previous assessments 
along with thb 1979-80 responses. A 5% 
subsample from each previous assessment 
was resccred, and percentages of agreement 
with the earlier scorings averaged 
approximately 99%. 

Most of the open-ended scoring effort was 
concentrated on a variety of exercises that 
required at le^st paragraph length written 
responses to poems and prose passages. 
Most of these items * ^re developed for the 
1979-80 assessment, although a f?w were 
1970-71 literature items readministered to 
measure changes in performance. Storing 
guides for these newty developed items were 
constructed using both 'field tryout'* (a 
prelirr..nary assessment to check the accuracy 
and effectiveness of items) data and actual 
assessment data. Scoring guides for the few 
reassessed items were developed in 1979-80, 
using both 1970-71 and 1979-80 assessment 
responses, to be consistent with the guides 
constructed for items first administered in 
1979-80. To ensure that scoring of the ^wo sets 
of assessment data was identical, all 1970-71 
responses to open-ended literature items were 
rescored at the same time that the 1979^80 
responses were scored. 



Although the use of a variety of types of 
tasks and scoring guides increases the 
expense and complexity of the open-ended 
scoring task, it neverth^ ':ss provides a more 
comprehensive means of assessing students' 
abilities to respond to written works. Five 
different types of open-ended exercises were 
included in the 1979-80 assessment of 
"response to written works." Each required 
different skills and levels of ability on the part 
of the respondents and therefore required a 
different scoring procedure. Each type of 
responding task and scoring procedure is 
discussed bnefly below and explained fully in 
this report and its appendicular materials. 

General responding tasks asked 
respondents to discuss the passage or poem 
presented, or to describe their thoughts or 
feelings about the text. It was expected that 
responses to this type of item would be highly 
text-dopendent and would allow the writer to 
select from a variety of perspectives. The 
writer was given very little explicit focus for 
his/her response, and was therefore free to 
choose whichever approach seemed most 
natural. The scoring guide categories for this 
type of exercise are descriptive and do not 
readily lend themselves to quantitative 
ranking. The response categories scored 
were: egocentric, personal, emotional, 
retelling, inferencing, generalization, analysis, 
reference to other works and evaluation, At 
age 17 only, three of these categories were 
further divided into two levels each to provide 
more qualitative information: analysis (level 1 

superficial, level 2 ^ elaborated): other 
works (level 1 ^ general, level 2 ^ specific): 
and personal (leve' * ■= global, level 2 = 
analytic). 

A second type of open-ended exercise is 
referred to as Inferencing, and Inferencing 
tasks required either general or specific 
inferences relating to the mood of, or a 
character in, the passage. Respondents were 
asked to describe the intent of the author, to 
describe the mood of the passage or to 
describe the character of the protagonist 
These kinds of items required the reader to 
interpret the passage and to explain the 
interpretation by relating it to the text. 
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Qualitative differences In score points were 
depenoont upon the identifiCv,tion of the intent, 
mood or character traits and upon the amount 
and nature of the supporting evidence 
provided, in addition, descriptive data were 
obtained concerning the source of the 
evidence— whether it related to the content or 
form of the text or whether it represented a 
subjective reactio.. on the part of the reader — 
and the number of pieces of evidence provided 
by the writer. 

Emotional rMpondlng tasks asked 
respondents to describe emotions or feelings 
aroused by the text. The scoring guide 
categorized both the identification of the 
emotion (or feeling) and the presentation of 
evidence supporting the emotion. Qualitative 
ranks were assigned and, again, additional 
descriptive information was obtained. 

Analytic responding tasks asked the 
respondents to analyze a passage or poem. 
Successful responses were those that went 
beyond a superficial interpretation and 
provided a theme or meaning for tho text, in 
addition, it was necessary that the 
respondents discuss the way in which some 
.*eature(s) of the text conti butes to the 
statement of the theme. These responses 
received only qualitative- rank scores. No 
further descriptive information^ was obtained. 

The final type of open-ended exercise, 
evaluativs responding tasks, asked students 
to evaluate F' poems or stories. . . 
Scoring guide > ese exercises measured 
the respondents abilities to state their criteria! 
and where apprcpriate. to provide examples 
from the text that are related to the criteria. 
Qualitative ranks were assigned to the various 
response types, and descriptive information 
was also assessed. 

Westinghouse Dat^Score Systems and 
National Assessme*-!! staff worked together to 
train readers. In training sessions, readers 
were given the scoring guide for an item and 
responses that exemplified each scoring 
category. The reasons why responses were 
classified in particular categories were 
discussed; scorers questions were answered, 
and, if necessary, modifications were made to 



ERLC 



scoring guides. Readers then scored several 
papers and categorizations were discussed. 
This process continued until readers were 
familiar with the application of the scoring 
guides and was repeated for each task and 
separate age group assessed to be sure that 
scoring was consistent. 

To further ensure the quality and 
consistency of scoring open-ended exercises, 
quality-control checks were conducted during 
the scoring of these exercises. At regular 
intervals, randomly selected responses were 
drawn from the total pool of responses for an 
item and read by randomly selected scorers. 
Both the responses and the scorers were 
selected without replacement; approximately 
10% of the responses were included in the 
quality-control check. Scores for the quality- 
control readers were recorded, and the 
responses selected for quality control were 
then put back into the total pool of responses 
to be scored during the regular course of 
scoring. Following scoring of all responses, the 
two scores for quality-control responses were 
compared. If discrepancies in scoring became 
apparent, scorers were retrained and, on some 
occasions, work was rescored. 

Percentages of agreement between 
quality-control and regular scoring were 
computed for each open-ended exercise. 
These data are summarized in Appandix B 

Meaiiar60 of Achievement 

The basic measure of achievement 
reported by National Assessment is the 
percemage of respondents answering a given 
item acceptably. This percentage is an 
estimate of the percentage of 9-, 13- or 17- 
year-olds who would respond acceptably to a 
given item if every 9-. 13- or 17-year-old in the 
country were assessed 

In addition to providing results on individual 
Items, National Assessment reports the 
average performance across groups of similar 
Items — for the learning area as a whole, for a 
particular theme, objective cr subobjective, 
and so on. These results constitute the mean, 
or arithmetic average, of the estimates of 
performance cn the group of items and are 
called the mean percentage correct. 

t 4 
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The items included in the calculation of a 
mean percentage usually are located in 
several exercise bool<lets and, thus, the mean 
percentage should not be construed as an 
average test score. 

To present a general picture of changes in 
achievement, National Assessment describes 
the gains or losses on a group of exercises in 
terms of the differences in the average 
percentage of acceptable responses. 

Unless the items summarized in the mean 
percentages of acceptable responses are 
identical, the means of one age group should 
not be compared with the means of another 
because their values reflect both the choice of 
exercises and the performance of the 
students. When only a few exercises are 
summarized by a mean, one should be 
especially cautious in interpreting results, 
since a small set of exercises might not 
adequately cover the wide range of potential 
behaviors included under a given objective or 
subobjective. The mean should be interpreted 
litera^V as the arithmetic average of the 
percentage of acceptable responses obtained 
from National Assessment samples on a 
specific set of exercises, ^t should not be 
construed as an average test score. 

The differences between perCi ntages or 
averages for a reporting group and that of the 
entire age group (nation) on an exercise are 
used to describe the performance of any 
reporting group relative to the entire age 
group This difference is a positive number if 
the group achieved a higher percentage or 
average than the entire age group and is a 
negative number if the group achieved a lower 
percentage or average. For example, a group 
performance of +1 8% indicates that the 
percentage of responses for the group is 1.8 
percentage points Higher than the national 
percentage of responses for that age level. 



astlmatlag VarlaHiUty 
Aclilavament Measures 

National Assessment uses a national 
probability sample at each age level to 
estimate the proportion of people who would 
complete an exercise in a particular way. The 
sample selected was one of a large number of 
all possible samples of the same size that 
could have been selected using the same 
sample design. Since an achievement 
measure computed from each of the possible 
samples would differ from one sample to 
another, the standard error of this statistic was 
used as a measure of the sampling variability 
among achievement measures from all 
possible samples. A standard error, based on 
one particular sample, serves to estimate that 
sampling variability. 

National Assessment has adhered to a 
standard convention whereby differences 
between statistics are designated as 
statistically significant at the .05 level of 
significance. That is, differences in 
performance between assessment years or 
between a reporting group and the nation are 
highlighted with asterisks only if they are at 
least twice as large as their standard error. 
Differences this large would occur by chance 
in fewer than 5% of all possible replications of 
our sampling and data collection procedures 
for any particular reporting group or national 
estimates. 

Organisation of Tliis Report 

The five chapters of this report are 
designed to both highlight findings and present 
detailed discussions of specific items. Each 
chapter begins with a brief discussion of the 
cluster of items designed to assess a specific 
objective and then moves to highlights of the 
results for that objective. Detailed findings are 
then reported by iterrji and age. Finally, each 
chapter, except the last one on general 
responding, ends with conclusions that 
summarize the findings in the context of 
implications'for teaching and learning. 
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Thd first chapter of the report begins by 
presenting a summary of the overall findings 
for the assessment and then moves to a brief 
but pointed discussion of implications for 
American schooling. Chapter 2. "Reading 
Habits, Interests and Attitudes. " presents the 
results of self-report questions designed to find 
out such things as where reading fits into 
students' priorities, how often students read 
and what they know abput their literary 
heritage. In addition, the chapter includes 
irtformation about students' literary knowledge 
and skills and how they have changed since 
1970. Chapter 3, "Explaining Responses to 
Written Works," examines how well students 
performed - n tasks that asked them to explain 
and substantiate their interpretations of texts. 
Chapter 4, ' Evaluating Written Works.". reports 
on students' abilities to evaluate writteri works 
against criteria. Chapter 5, "General 
Responding." presents the results of what we 
have learned about achievement in literature 
from the unstructured written responses of 13- 
and 17-year-olds. Appendix A lists members of 
the Reading/Literature Advisory Committee; 
Appendix B contains scorers' agreement 
percentages; Appendix C includes examples 
of items along with their scoring guides. 



A Vote About Znterpratatlons 
and Talao JuOlmeats 

Unlike other National Assessment reports 
that limit interpretive remarks to a particular 
chapter, this report includes interpretive 
remarks and value judgments throughout, due 
to the nature of the material. These comments 
represent the best judgments of the authors — 
Arthur Applebee, Kay Barrow. Rexford Brown, 
Charles Cooper, Ina Mullis and Anthony 
Petrosky — who are solely responsible for 
them. They do not necessarily reflect the views 
of the National Assessment of Educational 
Progress, the Education Commission of the 
States or the National Institute of Education. 
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CHAPTER 1 

Summary and 
ZmpUoatioiui 



The 1979-80 assessment of reading and 
literature was designed to provide a broad 
portrait of students' reading skiHs and 
attitudes toward reading. Many different 
formats, passage types and levels of difficulty 
were deliberately included to assess the 
extent to which such variations might 
influence students' performance. 




The chart following highlights some of the 
major findings discussed in this report. 



What Students Can Do 

1. Almost all students recognized the 
value and utility of reading. 



2. By age 17, most read a range of 
materials appropriate for their age level. 

3. Older students displayed stronger 
comprehension skills and were more 
versatile in writing about what they read 
than were younger students. 

4. By age 17, most students expressed their 
initial ideas and judgments about what 
they read, particularly when these 
involved personal reactions. 

5. Older students provided more evidence 
to support their assertions than younger 
students. 



Countervailing Tendencies 

1. Teenagers read little for their own 
enjoyment, spent more time watching 
television than they spent reading, did 
not read for long periods of time ^nd 
preferred movies to books. 

2. About 10% remained unable to read even 
simple materials. 

3. Older students displayed less 
commitment to reading than did younger 
students. 



4. Very few students at any age explained 
their initial ideas and judgments through 
reference either to the text or to their 
o^n feelings and opinions 

5. The evidence cited by older students 
does not reflect effective strategies for 
approaching a text, explanations 
remained superftcial and limited. The 
overwhelming majority of students ^ 
lacked strategies for analyzing or 
evaluating in the interest of deepening 
their understanding of what they read. 



The most significant finding from this 
assessment is that while students learn to read 
a wid6 range of material they develop very 
few skills for examining the nature of the ideas 
that they take away from their reading. Though 
most have learned to make simple inferences 
about such things as a character's behavior 
and motivation, for example, and could 
express their own judgments of a work as 
"good" or "bad," they generally did not return 
to the passage to explain the interpretations 
they made. 

In interpreting the results, we worked from 
a four-step model of the process through 
which comprehension would evolve. These 
four steps are: 

1. Initial comprehension, leading to 

2. preliminary interpretations, followed by 

3. A reexamination of the text in light of 
these interpretations, leading iO 

4. Extended and documented 
interpretation. 

Looking at the results across a wide range 
of items and tasks, our major conclusion is that 
America^ schools have been reasonably 
successful in teaching the majority of students 
to complete the first two steps of this model, 
but have failed to teach more than 5 to 10% to 
move beyond their initial reading of a text. 
Students seem satisfied with their initial 
interpretations of what they have read and 
seem genuinely puzzled at requests to explain 
or defend their points of view As a result, 
responses to assessment items requiring 
explanations of critena, analysis of text or 
defense of a judgment or point of view were in 
general disappointmg Few students could 
provide more than superficial responses to 
such tasks, and even the better responses 
showed little evidence of well-developed 
problem-solving strategies or critical-thmking 
skills 



These findings seem to us a direct 
reflection of current emphases in testing and 
instruction. In the classroom, teachers 
following traditional patterns of whole-class 
teaching and recitation move quickly from 
student to student so that many students can 
be involved without any one student 
dominating. The result is a pattern of teacher- 
dominated questioning in which brief 
comments from individual students are 
solicited and extended discussion is 
deliberately curtailed. Such techniques can be 
very effective in conveying an approved or 
conventional understanding of a difficult 
passage, but give individual students little 
opportunity to learn to formulate extended and 
detailed interpretations. 

The relatively short responses encouraged 
in classroom discussion parallel the multiple- 
choice and fill-in-the-blank formats that 
dominate standardized and teacher-developed 
tests. When doing well in most school contexts 
requires little beyond shor! responses, it is not 
surprising that students fail to develop more 
comprehensive thinking and analytic skills. 

More encouragingly, a large proportion of 
the students seem to be ready to learn how to 
explain and defend their initial interpretations, 
and through that process, to move to a better 
urrderstanding of the passage they are 
reading. By age 17, most of the students 
assessed are able to answer multiple-choioe 
questions requiring either literal or inferential 
skills. Most are also able to summarize 
passages and ^demonstrate the mechanical 
and grammatical writing skills they would need 
if they learned how to write more extended 
answers. 

What the majority seem to lack is- 
experience in undertaking such explanatory 
tasks and the problem- solving strategies and 
critical-thinking skills that would develop 
through such experience. They do not appear 
to have learned how to look for evidence for 
their judgments, whether by systematically 
analyzing some aspect of the passage or by 
referring to their own ideas and value systems. 
In examining students' responses to various 
items, we looked for a number of fairlv 
straightforward strategies that students could 



have used in answering the questions. These 
included such techniques as approaching a 
passage paragraph by paragraph (or stanza by 
stanza); focusing on one formal element at a 
time (e.g., imagery, theme); examining how the 
different sections interact around one 'main 
idea"; applying external frameworks for 
making sense of the passage (e.g., literary 
history or psychology); or comparing the 
passage with other works. 

We were not looking for "correct" 
approaches to particular passages; rather, we 
assumed that such systematic strategies 
would offer students alternative ways to" 
organize their explanations. What we 
discovered, however, was that students appear 
to have few strategies for approaching these 
tasks, relying at best upon a list-like citation of 
separate and unrelated bits of evidence for 
their opinions. Given the extent to which 17- 
year-olds were successful in formulating initial 
judgments anJ interpretations, we feel that 
with some guidance in how t j approach these 
tasks (and with better motivation), a relatively 
high proportion could achieve more adequate 
comprehension of the passages they read. 

Other rindlngt 

A number of other threads that run 
throughout the results are worth highlighting. 

• The nature of a particular passage has a 
strong, shaping influence on the 
characteristics of students' responses. 
Passages that interest or engage readers 
lead to fuller, rpdre elaborate discussion. 
Passages that are particularly difficult lead 
to synopsis or summarization, as readers 
grope for any meaning at all. Oth^r ^ 
passages drive readers toward evaluation, 
interpretation or personal associations. 

• Item formats also have a major influence 
on students' performance. Performance 
on mulliple-choice and short-answer 
items shows that by age 17 the majority 
of students were aWe to make accurate 
initial inferences about the passages they 
read. On the other hand, responses to open- 
ended items on the same and similar 



passafges highlight the limited nature of 
these comprehensive skills and make clear 
students' inabilities to elaborate upon the 
meanings derived from particular passages. 
Either item type alone would have provided 
only a limited portrait of students' 
achievement. 

• In general, students were better at 
discussing their personal reactions to a 
passage then they were at analyzing the 
passage itself. By age 17, the majority were 
capable of describing the feelings or 
emotions aroused by a passage, though 
they still had difficulty discussing mood, 
character or theme. Students focused their 
comments upon the content or "action" of 

a passage, rather than discussed aspects 
of form 

• Response patterns of 13- and 17-year-olds 
were similar in type and quality, with this 
exception: the younger teenagers were far 
more likely to make evaluative judgments 
about what they read. These judgments 
were seldom explained or supported at 
either age. 

• Between 1970 and 1980. both 13- and 
t7-year-olds became less likely to tfy to 
interpret what they r^ad and more likely to 
simply make unexplained value judgments 
about it. One way of characterizing the 
changfe during the seventies is to say that 
17-year-olds' papers became somewhat 
more like 13-year-olds' papers 

• Across tasks, girls read better than boys, 
and were also more successful in explaining 
their judgments. 

• As in Virtually all areas assessed by 
National Assessment, disadvantaged-urban 
students performed below national 
averages, while advantaged-urban students 
performed above national averages. 
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HHat Can Be Dona « . • 

The results summarized in this report 
suggest that American schoois have been 
successful at teaching students to formulate 
quick and short interpretations, but have 
not yet developed in students the skills they 
need to explain and defend the judgments 
they make. The end result is an emphasis 
on shallow and superficial opinions at the 
expense of reasoned and disciplined thought. 

If this emphasis is inappropriate— and we 
believe that it is— some restructuring of 
objectives and activities will be necessary in 
school programs. In particular, more situations 
must be created that require students to 
explain and defend their opinions at some 
length. Ideally, this would include both 
discussion activities, in which students have 
to contend with the immediate demands of 
an audience, and extensive writing, in which 
longer segments of text must be organized 
and related to one another. 

Neither suggestion is radical or unusual, 
but both require a realignment of resources 
and energies that may be difficult during a time 
when economic pressures are leading to 
increases in class sizes and in other aspects 
of teachers' workloads. The suggestions that 
follow Assume the vital importance of critical 
thinking, reading ar.d writing skills and seek to 
foster them within the constraints of finite 
resources and energies. 



...by School Administrators 

If students are to learn io engage in 
the kinds of extended inquiry and careful 
examination of evidence and opinions required 
in this assessment, the school administration 



must create an environment in which such 
inquiry is possible and valued. There are a 
number of aspects of the program that should 
be examined. 

• The testing program. Tests aie a direct 
reflection of what is valued by the school. If 
teacher-made tests, as well as standardized 
examinations, rely exclusively on short- 
answer formats, the message will be clear 
to teachers and students alike. Essay 
questions that require students to explain 
their points of view should be a regular part 
of any testing program. 

• Writing in the subject areas. The kinds of 
reading, writing and thinking skills stressed 
in this assessment are relevant in all areas 
of the curriculum. Teachers of all subject 
areas should be encouraged to include 
writing tasks as part of *heir courses, not to 
teach "writing " but to further learning of 
subject-area concepts. If writing is relegated 
only to the English class, it may seem of 
little importance to most students. 

• Systematic writing instruction. In addition 
to being required to explain their ideas 
and interpretations in a variety of subject 
areas, students need to be shown a variety 
of probiem-solving and critical-thinking 
strategies. Instruction in such sKillc} should 
be systematic rather than accidental, as part 
of the curriculum in English. 

• Institutional support for teachers. 

Incorporating new writing and discussion 
tasks into the curricuJum is time consuming, 
both in planning new activities and in 
reading the papers that result. Lower class 
sizes, released time or the provision 
of aides can help teachers manage the 
extra load. 

• Inservlce training. Most teachers have had 
no systematic training in the teaching of 
writing and thinking skills. The inservice 
program should provide opportunities for 
work in this area, for teachers of English as 
well as of other subjects. 
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The results of this assessment of reading, 
thinking and writing skills suggest that in many 
schools, instructional activities overemphasize 
immediate and sometimes superficial 
interpretations at the expense of more 
extended explanations and analyses. Various 
aspects of the instructional program need to 
be examined: 

• DItcuMlon periods. Does the pattern cf 
discussion allow individual students to 
state and defend their opinions? Or is the 
emphasis on developing a shared 
understanding under the teacher's guidance 
and direction? Small-group discussion 
may be needed to provide each student 
with opportunities to state and defend 
interpretations and opinions without using 
disproportionate amounts of class time. 

• Writing •ctlvitlM. Do students have 
regular opportunities to write at some 
length? Or does most written work focus on 
relatively short responses to study 
questions or worksheets? Students at all 
ages can be asliced to explain their 
judgments, though over the course of the 
school program the emphasis may shift from 
explaining personal reactions toward 

more formal analysis of texts and their 
meanings. 

• Problem-solving strategies. Do students 
have an opportunity to learn a variety of 
ways of analyzing a text in order to find 
evidence for their judgments? The most 
consistent weakness found in the present 
assessment centered on students' apparent 
lack of systematic app'-oaches to such 
tasks* In order to overcome this weakness, 
they need instruction in alternative 
approaches and practice in applying these 
alternatives in response to different texts 
and questions. 

• Textbooks. What kinds of reading, thinking 
and writing skills are stressed by the 
textbooks you are using? Can students 
working with the study questions provided 
stop after stating an initial opinion? Or do 
they have to find and organize evidence in 
support of what they have said or written? 



Many students believe they will emerge 
from school into an electronic world that will 
require little reading and less writing. Nothing 
could be further from the truth. Postsecondary 
education, whether academic or vocational, 
requires careful reading and strong skills 
in analysis, interpretation and explanation. 
America's work world, in which the 
"information business" is the fastest growing 
sector of the economy, also requires thoce 
skills. In a world overloaded with information, 
both a business and a personal advantage will 
go to those individuals who can sort the wheat 
from the feh^ff, the important information from 
the trivial. SktUs in reducing data, interpreting 
It, packaging it effectively, documenting 
decisions, explaining complex matters in 
simple rerms and persuading are already 
highly prized in business, education and the 
military and will become more $o as the 
information explosion continues They will also 
be increasingly important at personal and 
social levels. Quality of life is directly tied to 
our ability to think clearly amid the noise of 
modern life, to sift through all that competes 
for our attention until we find what we value, 
what wHI make our lives worth living. What we 
value is seldom on the surface and, when it 
is found, can seldom be defended from the 
incursions of the tnvial without sustained 
efforts to understand it more deepiv, to clartfy 
its nature and to explain it to ourselves and 
others. A society of individuals equipped to do 
this and aware of the values their forebears 
have left for them in their literature need not 
fear the future. A society m which the habits of 
disciplined reading, analysis, interpretation 
and discourse are not sufficiently cultivated 
has much to fear 

America s expenment in mass education 
has led to remarkable accompfishments, of 
which we all can be justly proud. However, the 
demands of the immediate future upon the 
technical, thinking, valuing and explanatory 
skills of this generation are such Ihat we must 
challenge our educational system to move 
beyond its already considerable achievements. 
We hope that the discussion in the following 
pages will represent a useful step toward 
defining that challenge. 
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Beading Habits^ Interests and Attitudes 




How well do America's students read? This 
report, in its entirety, represents an attempt to 
answer that often asked question, especially 
with respect to relatively complex kinds of 
reaamg7natefta^SuU> -itft^detailing students' 
achievement m this area, it might be usefurTo^ 
review students' reading habits, tastes and 
attitudes toward written works in order to 
establish a context for their performance. 
Where does reading fit into their priorities? 
How often do they read? What kinds of 
material do they prefer? What do they know 
about their literary heritage? What do they 
know about the terms and strategies used for 
understanding complex reading materials such 
as works of literature? 

A substantial part of the 1979-80 reading/ 
literature assessment was devoted to 
answering such questions. What follows is a 
synopsis of the results of those survey 
questions. 

How Muoh Do Thoy Talua 
aeadl&g? 

When asked ''How important is it to be able 
ta read?" over 95% of the students at each 
age said ''very important.'* 

However, when asked ''How much do you 
enjoy reading?" thay were less positive: 



Not only were fewer willing to choose the 
most positive response, but the percentages 
saying "very much" shrank considerably 
between ages 9 and 13 and reached their low 
point at age 17. 

A better indicator ofliow fnuclryoimg — 
people value reading is the extent to which 
they choose to read. Several questions 
surveyed the frequency with which they read 
and the priority reading takes vis-a-vis 
oth^r activities. 

More than two- thirds of the students said 
they read something at least once or twice a 
week for their own enjoyment: 



How Qft«n Do You Road For Your Own Enjoymont 
During Your Sporo TImt? 1979-80* 



AlmoM every day 
Once or twice a week 
Less than once a week 
Never 



Ago 9 

53 6% 
28 4 
12 3 
5 3 



♦Columns may not total 100% due to rounding 



Ago 13 


Ago 17 


35.4% 


32 7% 


35 9 


32 3 


20 9 


26 7 


76 


79 


rounding 


\ 

-f 



However, it appears that whet, they read, it 
is for less than an hour: 



How Much Do You Enjoy Roading? 1979-80* 

Ago 9 Ago 13 Ag# 17 

Vary much 8p 9% 

Somewhat 15 2 

Not at all 3.2 



49 8% 42 4% 

45 3 52.0 
4 6 53 



♦Columns may not total 100% due to rounding 



How Much TImo Old You Spontf Roading For 
Enioymont Yottorday? 1979-80* s 

Ago 9 Ago 13 Ago 17 



No timo 27,8% 

Loos than 1 hour 27 1 

1 or 2 hours 25 6 

3 or more hours 12 6 

•^oJgmns may not total 1C0% due to rounding or % 
not responding 



42 5% 
29 8 
20 6 
58 



43 6''/ 
32 2 
195 
42 
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Homework usually requires some daily 
reading. About two-thirds of the 13*year-olds 
and half of the 17-year-o»r s said they do 
homework of some kind; fewer than a third 
spent onl trourw^iwe oajt 



Mow Much Tim Old You Spond on Homtwork 
Ytettrdsy? 1979-tO* 





Age 13 


Age 17 


None 


30 2% 


31 7% 


Didn't do It 


6.2 


126 


Lees than 1 hour 


32 6 


23 8 


1 to 2 hours 


23 5 


22 3 


2 hours or more 


72 


93 


*Coi<jmns may not total 100% due to rounding 





In contrast, three-fourths of the 9- and 
13-year-old8 and two-thirds of the 17-year- 
olds indicated they watch an hour or more 
of television a day. About half the younger 
students and a third of the 17-year-olds appear 
to watch television at least three hours a day: 



How Much Time Did You Spend Watching 




TV Yeaterday? 1979-80* 










Age 9 


Age 13 


Age 17 


None or less than 








1 hour 


20 2% 


22 8% 


38 8% 


1-2 hours 


23 4 


28 2 


30 4 


3 or more hours 


501 


4; 9 


304 


'Columns may not total 100% due to rounding or % 


not responding 









Although television appears to consume 
much more of their time than reading, both 
television and reading have low priority 
compared to movies. Given a choice between 
going to a movie and reading, half the 9-year- 
olds and nearly two-thirds of the teenagers 
would rather go to a movie. The older they get. 
the less interest they expressed in reading: 



Age 9* Age 13 Age 17 

A Suppose you had several hours of free time and 
could do any of the following activities Which one 
would you enjoy doing the Most? 



Reading a book 


29 1% 


12 6% 


13.4* 


Watcning TV 


163 


20 2 


15.4 


Reading a magazine 


23 


4 1 


7.1 


Going to a movie 


51.4 


63.1 


63.6 


B. Which one of the 


same activities would 


you 


enjoy ^lofnglhe-beaet^ 








Reading a book 


17.9 


37.0 


46.0 


Watching TV 


19.5 


10.9 


20.8 


Reading a magazine 


48.8 


43.9 


27.2 


Going to a movie 


13.0 


7.8 


5.9 



*Columns may not total 100% due to rounding. 



Notice again how interest in reading 
decKnedesstudent&grew older This is further 
reflected in their response to the question 
"Have you ever felt bored when you read a 
story?;' The percentages replying affirmatively 
were 65% at age 9. 87% at 13 and 96% at 17. 

Almost all students said they have laughed 
while reading, arid 18% of the 9*year-olds, 
27% of the 13-year-olds and 39% of the 17- 
year-olds said they have cried while reading. A 
third of the 9-year-olds, half of the 13-year-olds 
and 70% of the 17-year-olds have felt angry 
while reading. 



How WaU Do Students Tblnk 
They Road? 

In answer to the question "What kind of 
reader do you think you are for your age?" the 
students responded quite positively: 



What Kind of Reader Art You? 197M0* 



A poor reader 


Age 9 


Age 13 


Age 17 


2 6% 


5 1% 


6.2% 


A good reader 


56 1 


65 8 


62 4 


A very good reader 


38 3 


25 4 


28 9 


1 don't know 


23 


34 


23 



♦Columns may not total 100% due to rounding 



Over 90% at each age believed they are 
good or very good. Results are somewhat 
different for several groups, however. For 
instance, a h i g h er p roport i on of f e ma le s find 
students in advantaged-urban schools 
believed they are "very good" reeders; lower 
proportions of rural students s»aid they are 
"very good," although the rural group 
performed about like the nation on reading 
^^^asseSsments. Black, Southeastern and 
disadvantaged-urban students, as groups, 
perceived themselves as somewhat better 
readers than their performance would indicate. 
Pe''haps this positive attitude accounts, in part, 
for the improved performance theise groups 
have displayed over the 1970s (Three 
National Aaaeaamenta of Reading: 
Changes in Performance, 1970-80, 1981). 

Large percentages of teenagers appear to 
have somo problems with thei*' reading, 
however. The following questions give us an 
indication of what students found difficult. 



Is each of the following things usually easy 
for you, or is it usually hard? 







Usually 


Usually 


No 






Easy 


Hard 


Response 


A Keeping your 


Age 9 


78 8% 


208% 


0 3%* 


mind on your 


Age 13 


67 1 


32 8 


0.1 


reading 


AOA 17 


63 1 




f\ A 
U 4 


B Reading long 


Age 9 


75 3 


23 8 


09 


sentences 


Age 13 


840 


159 


01 




Age 17 


81 0 


1 O 3 




C. Finishing 


Age 9 


601 


39 2 


07 


silent reading 


Age 13 


648 


35 1 


0 1 


in da&s in thA 


Age 17 


686 




U D 


amount of time 










given 










D Readina a 


Age 9 


42 0 


Of £. 


n fl 


Story with new 


Age 13 


51 2 


466 


0.2 


words 


Age 17 


57 3 


42 1 


06 


P Finishina 

1 II II9VIII lU 


Age 9 


79,8 


1 O 1 


1 1 


books vou have 


Aoe 13 72.7 


27,1 


02- 


started 


Age 17 


71.4 


27.7 


08 


F. Reading 


Age 9 


53 1 


463 


0.7 


books with 


Age 13 


67 4 


32 1 


0.5 


SFnflIt Dnntina 

9* 1 lOtI ^1 II llll .U 


Age 17 


71,0 




0 f 


Q Rndinga 


Age 9 


588 


40 3 


09 


book that 


Age 13 


505 


49 3 


0,2 


interests you 


Age 17 


548 


44 7 


05 


H. ReacWngvery 


Age 9 


47.4 


51 9 


07 


long books 


Age 13 


443 


55 6 


0.2 




Age 17 


41 5 


57 5 


1 0 


1. Finding a 


Age 9 


82 7 


17.0 


03 


book on a 


Age 13 


82 1 


177 


02 


subject that is 


Age 17 


81.6 


179 


0.6 


easy for you 










to read 










*Rows may not total 100% due to rounding. 





More than a quarter of the teenagers said it 
is usually hard for them to finish a book they 
h8'/e started; a third said it is usually hard keep- 
ing their minds on reading; and 40-50% said that 
it is usually hard for them to read materials with 
new words and to find books that interest them. 
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Wliat Kinds of Material Do They 
Like to Bead? 

Nine-year-olds liked to read books about 
real people and events most often; teenagers 
preferred fiction. Th« older teens read current 
news magazines and editorials more often 
than the younger teens. Relatively few read 
plays and poetry, and hardly any read literary 
criticism. 



Wliy Do Tbey Hepd? 

Students at all ages were asked several 
questions about the value of reading. The 
results indicate they know that one can read 
for a number of reasons but that they prefer 
utilitanan, practical reasons over reasons 
having to do with personal growth and 
pleasure. An example of the kind of questions 
thev were asked follows: 



Ptrctntagts Indicating Thty Often Like 
To Rtad Various Typtt of IMaterial 1979-80 



Biography 49% 



History 
Fiction 
Poetry 



47 
35 
35 



Age 13 

Fiction books 46% 
Short stories 46 
Nonfiction 

books 33 
Current news 



Plays 
Poetry 
Editorials 
Literary 
Cfrticism 



19 
14 

6 



Age 17 

Short stories 42% 
Fiction books38 
Current news 

magazines 37 
Nonfiction 
—books 
Poetry 
iditoriais 
Plays 
Literary 

criticism 



25 
17 
12 
11 



Age 9 Age 13 Age 17 



Reading can help me learn 

how to make things that t 

could use 
What I read tn books could 

help me understand more 

about the way t feel 

and act 
Reading could help me learn 

about famous people and 

important events 
What I read in stories or 

poems could help me find 

ways to get along better 

with people 



86 3% 91 



( 

4% 



92 1% 



65 0 63 2 71 1 



88 0 97 2 97 7 



53 4 60 9 65 8 



Preference for Fiction and Nonfiction* 

Age 9 Age 13 Age 17 

Never read during 

spare time 
Mostly read fiction 
Mostly read nonfiction 
Read fiction and 

nonfiction about equal 

^Columns may not Jotal. 100% due to rounding 



7 8% 


10 3% 


10 8% 


21 2 


30 7 


27 3 


23 4 


22 6 


30 5 


47 2 


36 0 


30 8 



In general, females read more fiction and 
poetry than males, but more teenage males 
read news magazines and editorials than 
females. 



When results for all items such as this one 
are summed, the following proportions 
emerge: 



Ways Reading Can Be Valuable: 





Age 9 


Age 13 


Age 17 


Source of mfonnatior^ 


82 0% 


84 4% 


91 8% 


Valuable for the culture 




68 1 


71 9 


Source of enjoyment 


75 2 


69 0 


66 2 


Promote personal growth 


54 5 


56 3 


67 3 


*Not asked at age 9 
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Most teenagers learn about current events from television and radio, although almost half appear 
to read a daily newspaper. Magazine reading for news is relatively infrequent. 



How On«n Do You Find Out About th« N«ws From 
Each of th« Following Sourctt? 







Evtry Day 


Television 


Age 13 


68 3% 




Age 17 


65 8 


Radio 


Age 13 


50 6 




Age 17 


686 


Magazines 


Age 13 


39 




Age 17 


26 


Newspapers 


Age 13 


41 1 




Age 17 


483 



*Rows may not total 100% due to rounding 



Savaral 


Savtral 


Savaral 




Tln»at a 


Times a 


Timas a 




Waak 


Month 


Yaar 


Navar 


26 0% 


3 5% 


0 9% 


a8%» 


29 4 


3 5 


07 


0.4 


33.9 


89 


26 


2.4 


25 5 


40 


1 1 


06 


22.8 


40 0 


142 


17.4 


25.5 


50 6 


134 


7 3 


35.4 


11 8 


52 


5 1 


33 6 


12 8 


2 4 


1 6 



In Summary: Students' Reading 
Habits aSid AtUtudea 

a They think they are good readers. 

a About 10% at each age do not read at all 
in their spare time (they may be able to, 
but they do not choose to). 

a Abou^ one student in six, at each age, 
has difficulty finding books that are easy 
to read. 

a A third to more than half the teenagers 
have problems concentrating on their 
reading^ finishing classroom silent 
reading in time» finding books that 
interest them and reading long books. 

a They believe reading is important and 
enjoy it at least "somewhat." However 
less than half of the 13-year-olds and 17- 
year-olds enjoy it "very much." 

a They like reading less as they get older. 



a They watch television far more often 
than they read. 

a When they read, it is for short periods of 
time. 

a They prefer movies to either television or 
reading. Almost half of the 17-year-olds 
selected reading a book as their least 
favorite, choice. 

a They see reading's value to lie primanly 
in its being a source of information, not a 
source of enjoyment or self- 
understanding or cultural values. 

a In general, females read more than 
males, white students more than blacks, 
advantaged students more than ' 
disadvantaged. 

In conclusion, students appear to feel that 
reading is valuable and useful in general 
terms, but have only a moderate interest in 
reading themselves. 
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What Do Thay Know About 
Ziitarary Worka and Oliaraotara? 

The first National Assessment literature 
objectives stressed the importance of a firm 
grounding in classic literary works (Literature 
Ob|ectives, 1970). Attempts to assess this 
area have had limited success, however, for 
two major reasons, it is difficult to get a 
consensus about what works are "classics/* 
and it has been impossible to sample a 
sufficient number of works to be able to draw 
conclusions about the breadth of students' 
exposure to literature. Consequently, 
our information does not lend itself well 
to generalizations. 

Teenagers' exposure to some literary 
works was assessed by presenting them with 
"disguised" myths and stories. Here is a 
typical exercise of this type: 

lilatan o^refUlly when I r«Ad the pASftage whloh 
la baaad on a famous story, then All In the oTal 
beside the name of the story that you tlilnk 
the passage Is based on. 



Here is another approach: 

''Let's call our motel the , so that 

travelers will know they can have a ^ood long 
rast." 

Brom Bones 
Ichabod Crane 
Davy Crockett 
Rip Van Winkle 
I don't know. 

The exercises used are included in the 
Reading/Literature Released Exercise Set. 
1979-80 (1981, pp. 260-280); the works or 
characters involved follow. 



Works or Chsrscters Assessed, Agss 13 and 17 


Ths Emperor's New Clothes 


Tom Sswyer 


Ssmson 


Noah 


Johnny Appteseed 


Qalshad 


Chicken Little 


John Henry 


Rip Vsn Winkle 


Robin Hood 


Adsrr 


Cairi and Abel 


Job 


Faust 



Old Ptterr«n hsd hsd a good life, a good farm —the 
richest in u.. country, a happy family, a nice tidy 
Income. Iven eo, he wasn't proud. Just thanXful 
Then the locusts came and ate his wheat, someone 
poisoned hie welle, and to cap It off, hie children 
died of diphtheria. What had happened, f etereen 
wondered What had he done? It was enough to 
make a man lose his fklth 



Which one of these stories do you think ,ue passage 
Is based on*^ 

Job 

Exodus 
Barabbas 
The Fall of Man 
I don't know. 



When results for these exercises are 
averaged and the averages for both 
assessments are compared, we find a drop 
in recognition at both ages: 



Mean Percentages Recognizing LItersry Works end 
Chsrscters 



1970-71 ISTMG Change 



Age 13 
Age 17 



65 2% 
70 5 



59.2% 
64.7 



-5.«* 



^Asterisk indicstes significant chsnge in performsnce 
between assessments at the .05 level 
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A handful of exercises assessed 
recognition by presenting students with 
parodies of such works as "Old Ironsides/* 
"Sea Fever." "Casey at the Bat" and "Paul 
Revere's Ride" and aslcing them to recognize 
the works being parodied. Percentages of 
9-year-olds recognizing the works were 
the same in the 1970-71 and 1979-80 
assessments. However, the average 
percentage of success for 13-year-olds 
declined b points and the average for 17- 
year-olds declined 12 points. 



A number of questions tested 13- and 
17-year-okls' understanding of puns, similes, 
hyperbole, theme, genre and other such 
aspects of literary language. The average 
percentage of success for 
13-year-olds for 13 such questions was 72%; 
the average for 17-year-olds on the same 
questions was 85%. Clearly, a majority of 
teenagers could recognize and employ such 
terms when given an example and asked to do 
so. 



Mesn f^csfitsgM HMognlzIng Parodist 

lerim 197M0 ChMigs 

Aqs 9 42,3% 41.1% - 1.2% 

Ag« 13 53.4 46.4 — SO* 

A0* 17 68.7 56.8 -11.9* 

•Aattritk indicatst significant change in parformanca 
batwaan asaaaamanta at tha .05 laval. 



Because the number of works and 
characters assessed is so small, it is difficult 
to My what these drops mean about exposure 
to literature. The declines may imply that 
exposure is less now than it was a decade 
ago, but we would need far more information 
to be sure. 

mkmt Do Thmj Know About 
LiUTTy LaiifMlo and 
OoiiToiitioiuir 

Another dimension of literature studies 
is understanding imaginative language. 
Students* understanding was tested largely 
by asking them to write about literary works 
(Me Chapters 3 and 4), but they also were 
given some multiple-choice items in both 
MMMfnents. Several questions about 
metaphors revealed an improvement in 
understanding at age 9, but neither 
Improvement nor decline at the other two 
agM. Although this finding rests on only four 
items, it is consistent with the general gain in 
9-year-olds* reading ability reported elsewhere 
(ThrM NatioMi AteeMiMfita of RMding: 
ChangM in Performance, 197M0, 1981). 



In Summmryi Stndonts' 
Kaowlodge of Xdterury Works 
and OoiiTaiitloiui 

• Although assessment of characters and 
works was skimpy, there is no reason to 
believe the students have read broadly, 
and there is some reason to believe their 
exposure to literary works has declined 
over the last 10 years. 

• When given specific examples and 
directions, most students could Wentify 
instances of metaphor, puns, hyperbole, 
and so forth. Their skill in this does not 
seem to have declined. 

The best test of their knowledge of 
literary conventions is to see what they do 
when asked to analyze a work or explain 
their responses to it. The rest of this 
report addresses that issue along with 
many others. 
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aaq^lalning Besponses to 
Written Works 



In addition to initial comprehonsion of 
written texts, the objectives for the 1979-80 
assessment of reading and literature stressed 
students' abilities to respond to texts in 
interpretive and evaluative ways. Such skills 
are more than the tools of the literary critic; 
they are powerful ways in which initial 
readings can be modified and extended 
through careful consideration of the text itself. 

A variety of specific exercises was 
developed to assess the extent to which 
students could explain and extend their 
interpretations. 

Although responses to this set of tasks 
varied from text to text and item to item, 
each task drew upon a set of related 
comprehension, thinking and writing skills. 
Each task began with a test of initial and 
relatively general comprehension of a text, 
whether storV or poem. Drawing upon this 
initial comprehension, students then had to 
summarize some aspect of it— to identify the 
theme, for example, describe a character or 
label their own feelings about the text. The 
next step in each of these tasks required the 
readers to return to the work itself, to explain 
how the details of the story or poem joined 
together to reinforce the interpretation that 
they made. 

As readers try to account for specific 
details in terms of their initial interpretations, 
they often find that the text is more 
complicated then they had originally 
recognized, and its message is accordingly 
richer than it appeared at first. This richness 
may in turn lead to modifications in the 
initial interpretation, or at least to a fuller 
understanding of the ways in which that 
interpretation (whether of character, theme or 
mood) is supported by the text itself. 



The steps that underlie these tasks, then, are 

1. Initial comprehension, leading to 

2. Preliminary interpretations, followed by 

3. Reexamination of the text in light of 
these interpretations, leading to 

4. Extended arrd documented 
interpretation. 

In order to respond successfully to this 
set of tasks, students needed systematic 
procedures for approaching ». text — 
procedures as simple as looking in turn at 
successive stanzas in a poem, discussing a 
list of character traits one at a time or tracing 
an image from beginning to end. Often, the 
more successful responses reflect knowledge 
of specialized analytic vocabulary and 
concepts, terms such as ''imagery," 
"metaphor" and "point of view." It is the 
systematic application of such concepts in 
approaching the text, however.'that seems 
crucial; rote knowledge of technical terms 
does little to enhance understanding of 
reading matter. 

Students whose responses reflect 
systematic approaches to what they read often 
seem to emerge with a better understanding 
of what it is about. Much as the objectives for 
the assessment imply, in explaining and 
justifying their initial response to a text, they 
moved beyond that response to new layers 
of meaning that their peers often seem to 
have missed. 
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When we look at what students did when 
they- were told to analyze texts, what do we 
see? First, we notice that whether the task is 
to analyze a text for mood or character or to 
analyze a text for theme, students at ages 
13 and 17 performed In similar ways. Their 
analyses, even the very best ones, were 
heavily weighted by prelimin^iry statements on 
theme, mood, character or emotions. A typical 
paper, no matter how buphistlcated, began 
with a statement like: 'The theme of this poem 
is to be yourself,** or "I pityed (sic) the dog 
because it was half blind, and had mange," 
and then went on either c>to expound on the 
idea in the style of someone who sees it in 
only one dimension or to relate the idea back 
to the text by pointing to one or two pieces of 
evidence that clearly, but superficially, support 
it. Generally the responses were short and 
superficial with little evidence to support 
the analyses. When we look at the data for 
17-year-olds discussing the theme of the 
poem, '1 was you," we see the typical pattern: 
only 4.6% of the students provided substantive 
evidence to support their analyses; 19% gave 
minimal evidence of analysis and 578% 
offered a synopsis or brief statement of theme 
with no supporting evidence. 

Read the poem below. Then write an essay 
about an important idea or theme of the poem. 
In your eftsay tell how suoh thinge as the 
images, events, sound and struoture contribute 
to this idea or theme. We are Interested In 
what you have to say, not your spelling and 
pimotuation. Write your essay on the lines 
provided on the next two pages. You will have 
9 minutes to read the poem and write 
your response. 



i waa you 

1 smiled 

your smile 

till my mouth 

was set 

and my taoe 

was tignt 

and it wasn't right 

It was wrong 

1 was you baby 

1 was you too long 

1 said 

your words 
till my throat 
closed up 
&nd 1 h&d 
no 70loe 
and 1 had 
no oholoe 

but to do your song 
1 WES you baby 
i was you too long 

1 lived 

your lifiB 

till there was 

no me 

1 was nesh 

1 was hair 

but i wasn't there 

it was wrong 

1 was you baby 

1 was you too long 

and baby baby 

the worst thing 

to It 

is that you let me 

do it 

80 who was weak 

and who was strong 
for too long baby 

Dorl Prsvln 



And while upwards of 75% of the 17-year- 
olds showed comprehension of some kind on 
multiple-choice items for these texts, only 5 to 
10% showed strong analytic sicills. Another 35 
to 50% showed some uneven evidence of 
Icnowing that they can lool( at a text 
analytically, though they might not do so with 
any detail. Another 25 to 40% primarily retold 
the text in some way, rather than used it as 
evidence to substantiate ideas or opinions. 
About 10 to 15% showed no evidence of being 
able to do the analytic taslcs at all. 
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The exceptions lo this general trend are 
noteworthy though, because they point to the 
power of the text to influence stuoents to'write 
more involved, substantiated, elaborated and 
coherent analyses. The best example comes 
from both 13- and 17-year-olds' responses to 
the highly emotional selection "A Story of a 
Good Dog" (Appendix C). The specific task 
asked readers to use the text to explain their 
feelings and emotions about the story. Sixty- 
two percent of the 13-year-olds and 58% of the 
17-year-olds wrote adequate analyses— that 
is, analyses that used some reference to the 
text in order to explain the student's response. 
Typically, both 13- and 17-year-olds wrote 
better substantiated analyses in response to 
"A Story of a Good Dog ' story than they did in 
response to other stories. However, very few of 
even the better analyses proceeded in any 
way that would allow us to conclude the 
st:*dents had a procedure for approaching this 
kind of task. We would have expected, for 
example, that students might have proceeded 
simply by analyzing paragraphs or stories 
oqe at a time; or, when given the general 
directions to use plot, character or language 
as they contribute to a theme, they would have 
proceeded by looking at these elements in 
relation to the theme. No such demonstration 
of tystematlc approaches to analyMS 
showed up in the reeponees at any age 
group. 

We did see. however, that generally when 
students used evidence, most focused on 
content. Only 5% or less of the students 
mentioned any aspect of the form of literary 
selections, and only 10% or less cited personal 
evidence. This picture changes slightly for "A 
Story of a Good Dog," with 76% of the 13- and 
17-year-olds using evidence from the story to 
substantiate their analyses and 9% of the 13- 
year-olds and 12.4% of the 17-year-olds using 
personal evidence. Again, it seems that the 
text ("A Story of a Good Dog") and the task (to 
discuss your emotions rather than character, 
mood or theme) drove the students to new 
approaches. Typically, though, here again, 
only 1 % of the students discussed any aspect 
of the form of the selection. 



9-Year-Olds' Succeat In 
Expialning Responses 

The ability to explain responses through 
careful reference to a text is a relatively 
sophisticated one that develops throughout the 
secondary school years. For the assessment 
of 9-year-olds, two items were used to provide 
some indication of the ability of younger 
students to undertake such tasks. One item 
asked readers to explain their feelings and 
emotions in response to the poem "Old Dog." 




Read the poem below and then answer the 
questions on the next two pages. 



The other asked them to explain a 
judgment of William Carlos Williams' intention 
in writing "As the Cat." 



oia Boc 

Toward the leist In the morning she could not 
get up, even when I rattled her pan. 
I helped her Into the yard, but she stumbled % 
and fell I knew It was time 

The last night a mist drifted over the fields 
In the morning she would not raise her head— 
the far, clear mountains we had walked 
surged back to mind 

We looked a slow bargain- our days together 
were the ones we had already had 
I gave her something the vet had given, 
and patted her still, a good last friend 
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Here la a poem about whloh you are going to 
be a«ked two questions. I wlil read the poem to 
you as you read It to yourself. When we have 
read the poem oarefully, I will read you the first 
question and you are to fill In the oval beside 
the answer you think best. Then I will read the 
^cond question to you and you are to write 
your answer in the answer space. 



Table 3-1 summarizes the major results. 



As the oat 
olimbed over 
the top of 




the Jam closet 
first the right 
forefoot 




carefully 
then the hind 
stopped down 




into the pit of 
the empty 
flowerpot 


William Carlos Williams 



TaM« 5-1. 


^•re«Btag«s axplainlng m«s9oas«si Mgm 9, 


1980 








Berely 
Adequete 


Adequate 


Inedsquete 


Unretebte 


Feelings or emotions 
Old Dog 


15,7% 


39 0% 


33 1% 


12 2% 


Author s intent 
As the Cat 


11 5 


3y6.3 


39 8 


124 



Although all but about 12% of the students 
attempted both tasks, their responses were 
very limited. For example, some 88% were 
able to describe their feelings in response to 
the poem "Old Dog. " but only 16% adequately 
related these feelings to characteristics of the 
poem (see Appendix C for scoring criteria). 
Thirty-nine percent managed a barely 
adequate response, briefly citing a single 
aspect of the poem as the justificat.on for the 
feelings aroused. (Typically, with this poem, 
9-year-olds claimed to feel "sad — because 
the dog died ") With both poems, students 
turned for evidence to the specjfic content (or 
"action") of the poem, rather than commenting 
upon formal characteristics of the text The two 
examples below illustrate different ways in 
which 9- year-olds attempted to relate their 
feelings to the content of the text 



I felt iclnd Qf sad for the dog's death. 

It was that the poor doggy st^umbled; and on 
the next morning he wouldn*t ralae hie head at 
all; and when the dog died, the girl patted the 
dog on the head and that was what made me 
feel the vay>eald up there 

I had two feellngsM^was sad and happy. 
I was sad because the dqg was going to die. 
And I was happy beoause th^^^elped the dog. 

"Old Dog" also prompted some personal 
associations from 8% of the students, usually 
in the form of anecdotes about their own 
experiences with death. 

Though the 9-year-olds' data are very 
limited, the results suggest that most 9-year- 
olds hav6 yet to develop the skills necessary 
for explaining their interpretations of text, 
though they are quite capable of forming initial 
impressions about meaning. 
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13-Year-Oldt' Success in 
Explaining Responses 



For 13-year-olds, similar sets of items asked for text-based explanations of interpretations of 
character, of mood and of feelings and emotions. Table 3-2 summa.izes the results. 



Tsbla s-a. 



Vnfntmfa ■xpl«lnln< «e»pon»e»; Af 15, 1979* 





Adequate 


Bartly 
Adequate 


Inadequate 


Unratabl« 


Character 










"Somebody's Son" 
Unreieased description 


27 0% 
21 3 


45 6% / 

46 4 


21 6% 
29 0 


5 8% 
34 


Mood 










"The Closing of the Rodeo * 
Unreieased narrative 


32 2 
35 4 


32 0 
24 5 


28 5 
34 5 


73 
56 


Emotion 








'Old Dog 

"A Story of a Good Dog " 


40 5 
61 8 


36 7 
16 5 


19.4 
104 


3.4 
11.2 



•Rows may not total 100% duo to rounding 



Like the 9-year-plds, the 13-year-olds were 
quite capable of making initial generalizations 
about these aspects of text. Between 89 and 
97% of the students attempted a respor>se to 
these items, identifying some aspect of 
character, mood or emotional response. 

At least rudimentary explanations of these 
interpretations were offered on 60 to 78% of 
the papers. Items discussing personal feelings 
if) response to a text prompted somewhat 
y^more adequate responses than those involving 
discussion of character. Discussions of mood, 
which are related to personal responses but 
mor0 abstract, seemed of intermediate 
difficulty. Fewer of the l"&-year-olds moved 
beyond a barely adequate explanation. 
Though 62% were able to explain their feelings 
about "A Stoiy of a Good-Dog/' only 21% 
were able to provide adequate explanations 
of their characterization of the old man in 
an uareleased exercise based on a 
descriptive passage. 



Even the better responses at age 13 
tended to be short and based on limited 
evidence. For most items, fewer than 5% of 
the responses drew more than three bits of 
evidence from the text; the major exception 
was in response to "A Story of a Good Dog," 
where fully 20% of the 13-year-olds elaborated 
their responses with four or more bits of 
evidence. References to specific aspects of 
content dominateo resoonses. although a few 
references to form appeared in response to 
the two unreieased exercises (3% of the 
students, in both cases). Personal reactions 
were also cited by a number of students, 
depending upon the nature of the text. These 
ranged from 1% in response to a relatively 
remote situation in the unreieased exercise 
asking for characterization, to a high of 18% in 
discussing personal feelings aroused by the 
death in the poem "Old Dog." 
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The responses below are typical of those in 
which students directly associated their own 
experiences with those in the poem. 

I felt, sort of flftd but aooeptaMe. 

I Ilka ftnlmalfl eBpeciAlly doga. I have & 10 
yaar old poodla &t homa and I know that ha is 
not going to llva foravar so tha poam gava ma 
aomathlng to nzpaot. Tlila poam made ma gat a 
faellng that I should ^oapt an old dogs daath 
but I don't think it is going to ba as aasy for me. 



My amotions ware sad and sorriful 

It made ma faal that the daath of my dogs 
will coma. 



It made me think of whan my dog died 
Uy d og dlad over night ha was sorta lik e 
that dog 



17-Ye«r-0ld8' Success in 
Explaining Responses 

Items at age 17 paralleled those at age 13, with the addition of a fourth set asking students to 
identify and explain (with evidence from the text) an important idea or theme they saw in the 
selection. Since theme is a relatively abstract and difficult notion, the question stems for these items 
were highly specific, pointing the students toward such differing aspects of the text as plot, 
characters, setting, images, language and structure for evidence. Results fiom all items are 
summarized in Table 3-3. 



Tmhlm 8-8. 



Vereeatafes Bxplalnlag mesponsest Jk|# 17, X9ao* 



Thame 
" I wss you" 
Unreleased nsrrstive 

Character 

"Somebody's Son * 
Unreleased description 

Mood 

"The Closing of the Rodeo' 
Unreleased narrative 

Emotion 
"A Story of a Good Dog ' 



Adaquata 

4.6% 
97 

414 

36 3 

41 2 

37 6 

57 7 



Baraly 
Adaquata 

19.0% 
20 2 

42.4 
43 1 



37.9 
26.6 

22.9 



Inadaquata 

70.3% 
61 5 

136 
16.3 

166 

32.2 

11-6 



Unratabla 

6.1% 
6.6 

2.5 
2.3 

43 
34 

77 



*Row$ may not total 100% due to rounding 
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At age 17, the ability to analyze a text for 
evidence to support preliminary interpretations 
varied sharply with the nature oJ that initial 
interpretation. On the one item asking for 
explanations of feelfngs. over 80% of the 
students wen) able to provide at least 
minimally adequate explanations, and 58% 
elaborated on their responses. At the other 
extreme, fewer than 30% of the students were 
able to analyze a text in order to provide 
even minimal explanations for generalizations 
about theme or main idea. The evidence 
they provided tended to involve a fairly simple 
and direct line between a statement about 
[le and the "act" in the text, as in the 



followiflg^ejcamples. 

I feel the idea orthiaj)oein la to po int out 
that, everyone has theU owft41g, ar the person 
In thle poem la Uvln< someone elie^lfe, not 
hie own, now he or she realises that fch^^n't 
Uying their lift the way they wjmt to, they're^ 
llTlni their life the way aomeohe else wants 
them to live. 

It aay to be youraelf, and don't try to aot Ilka 
anyone else or you will beoome that peraon. 
Also It eaya that you ahould not let aomeone 
ImlUte you. There doee not need to be two of 
one peraon In thla world. Thla peraon Imitated 
the aeoond pereon ao long, he loat all aelf 
Identity He had loat hlmaelf too long y oould 
not nnd hlmaelf. 

In spite of the fact that students were 
directed to look for and use types of evidence 
for this task, only 5 to 10% of them connected 
their generalizations to various aspects of the 
text. Here are examples of the few successful 
responses. 

I think the theme la one of hopelesaneaa. 
The "1" of the poem aeema hopaleaaly compelle d 
to be the ''you." The poem aeema to indicate a 
great love or reapeot. Why elae would "1 lived 
your Ufa ml there waa no me . . . "? 

The etruoture of the poem, the amall-oaae 
letters, no punctuation, the ahort phraaee, tit 
perfaotly with the theme. There la no real order 
or aenae to It, but It oontlnuea on. 

I pan almost hear the poem t>elng read In a 
•ort of haunting, whlapertng tone. 

I ftel that the whole poem la very efftotlve 
•nd I enjoyed reading it lb a amaU extent, I pan 
relaU to the need to be like acme one to the 
point of loaing your own identity. 



The whole theme of ^he poem In my opinio n 
la. that the auther la trying to aay that you can't 

Uve your life through other people. 

I and waa are uaed continually throughout 
the work. Waa la uaed to ahow that no longer 
am "I your baby . " Alao the llttie 1 la Juat kind of 
aaylnn that it'a not myaelf but what you want 

me to be. 

"1 waa your baby" and "i waa you to long" la 



alao repeated. Thla repetition helpa to ahow 
that ahe'B not her oaby any longer and ehe/he la 
going to live her own life now. 



In explaining character, mood and feelings 
aroused in response to a text, 17-year-olds 
continued to turn primarily to aspects of 
the content or action in the texts. A small 
proportion of the students (5% or less, 
depending upon the specific text) also made 
some reference to formal characteristics. 
Between 3 and 12% drew upon personal 
reactions as evidence for their interpretations. 
Discussions of theme, where specific types of 
e\^rdeQce were highlighted by the instructions, 
were not Categorized according to the types of 
evidence used?""--.^ 

Overall, the results suggesTlhat 17- year- 
olds are unused to being asked to ex^taia 
the meaning they draw from texts. Althougli" 
a reasonably high percentage defended 
assertions about such limited aspects of text 
as mood and characterization. Ihey had no 
systematic way to go about analyzing a text as 
a whole. The most frequent approach when 
confronted with a task asking for a discussion 
of the theme or main idea was to provide a 
summary or synopsis; 47% of the responses 
to "i was you" and 49% of those to the 
unreleased exercise took this approach. 
The responses below represent the approach 
taken in nearly half of the responses to 
"i was you." 

The peraon aeema to think that the other 
peraon haa led them away, brought them deeper 
and deepe r into love. I'hey aeem to have let go 
and now realise how It waa going to hurt when 
It waa all over between them. They told how the 
different thing attraoked them and made It 
woree« The peraon aeema to be trying to aay 
that they have lived the othera live but nothing 
oame In return to make them a better peraon. 
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This poem is dlscrlblng someone who has 
loat their identity. They have tryad to be 
aomaona else bo long They've forgotten who 
thay are. In the first of the poem the parson put 
on the appearance of someone else &nd In doing 
«o forgot what they really looked lilce. Next, 
thay began to talk Ilka the parson and now ha 
doaan't know what hb had to say and last he 
began to live like that person and in doing eo he 
gave up his own life. 

Although such a rastatdment of the events 
in a text can be a productive beginning point, 
the majority of the 17- year- olds did not move 
beyond this initial rttelling. 

When we look at our open-ended 
inferencing data by groups — race and 
community— we see, generally, that black 
9- and 13-year-old students performed at least 
10 percentage points lower than their white 
counterparts, while black 17-year-olds 
performed from 15 to 2b% below their white 
counterparts. Thirteen- year- olds from 
disadvantaged- urban communities performed 
7 to 33 percentage points below students 
from advantaged- urban communities, while 
students from rural areas performed slightly 
above students from disadvantaged 
communities but 6 to 17 percentage points 
below students from advantaged-urban 
communities. 

Seventeen-year-old students from 
disadvantaged- urban communities performed 
12 to 27 percentage points below students 
from advantaged- urban communities, while 
students from rural areas performed 
considerably above students from 
disadvantaged communities but, with two 
exceptions, 10 to 15 percentage points below 
students from advantaged-urban communities. 

These differences between black students 
and white students and students from different 
types of communities are somewhat larger 
than the differences found when we examine 
performance on multiple-choice questions/* 



^tae Thraa National Aeaaasmantt of Reading: 
Changat In Parformanca, 1970-80 (198>) 



Changes In Analsrtlo SkiUs 

Two released items^from the 1970-71 
assessment of literature were readmmistered 
for this assessment; together with the multiple- 
choice items, these items formed the basis for 
our analytic task change data. 

One task asked 9^year-olds to identify and 
substantiate their claims about William Carlos 
Williams' intention in writing the poem "As the 
Cat." Results showed virtually no change from 
^97^ to 1980. in 1971, 77.8% of the 9-year-olds 
answered the inferential comprehension 
multiple-choice item correctly, while 78,7% 
answered the same item correctly in 1980. 
Twelve and three-tenths percent of the 1971 
9-year-olds wrote adequate analyses of the 
text for author's intention, while 11.5% did the 
same task adequately in 1980. 

A somewhat different picture emerges from 
the change data on 13- and 17-year-olds. 
These students were given a poem, "The 
Closing of the Rodeo'' by William Jay Smith, 
and were asked to identify and substantiate 
mood. The identification of mood was handled 
through a multiple-choice item, and the 
substantiation was handled in an open-ended 
analytic task similar to the other analytic tasks. 

Here Is a poem about which you are going to 
be asked two questions. I will read the poem 
aloud aa you read it to yourself. ?rhen we have 
read the poem carefully, I will read you the first 
question and you are to fill in the oval beside 
the answer ^ou think best. Then I will read the 
second question to you and you are to write 
your answer in the answer si^ace 



The Closing of tho Rodoo 

The l&rl&t snaps, the cor^boy rolls 
His ptoK, and mounts and rides away 
Back to the l&nd the cowboy goes. 

Plumes of smoke from the factory sway 
In the setting sun The ourtaln falls. 
A tra^n In the darkness pulls away, 

Goodbye, says the rain on the iron roofs 

Goodbye, say the barber poles 

Dark drum ^he vanishing horses' hooves 



William Jay Smith 
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On the multiple-choice item, 13-year-dds 
exhibited a significant 4.5% decline over the 10 
years. On the open-ended analytic taslcs. on 
the other hand, the 13-year-olds' performance 
remained constant over the 10 ysjrs. In 1970. 
33.2% of the 13-year-olds wrote adequate 
analyses, while in 1979, 32.2% did so. 

The r\»sults of the mutiple-choice item for 
17-year-olds — a 2.5% decline over the 10 
years ^ are similar to the mean declines 
reported in inferential comprehension for 17- 
year-olds on 10 years of change in reading.^ 
Even more remarlcable is the drastic decline 
of 10 percentage points for the number of 
17-year-old8 writing adequate analyses for 
the open-ended task that asked them to 
substantiate their claims about the mood by 
turning bacl< to the *«xt for evidence. In 1971. 
51.2% of the 17-ye 3lds wrote adequate 
analyses, while in 1^80. 41.2% did so. Of 
the 17-year-olds. 11.1% wrote inadequate 
analyses in the earlier assessment, while 
16.6% wrote inadequate analyses in the later 
assessment. No evidence at all was given by 
12.8% to substantiate their claims for mood in 
the earlier assessment, while in the later 
assessment. 18.2% gave no evidence. 

The convergence of the reading and 
literature results for 17- year-olds is fairly 
compelling: inferential comprehension 
declined significantly for this age group on 10 
years of reading change data, and the same 
trends are evident for this age group on these 
reading, responding and writing taslcs. In 
other words. 17-year-olds' performance on 
inferential taslcs declined in all areas of 
reading and literature during the 9-year period; 
and the decline was most drastic when the 
students were asked to explain and 
substantiate their responses. 

When we \ook at our change data by 
groups — e.g., race and sex — we see an 
uneven picture of gains and declines. At age 9. 
students from advantaged- urban communities 
declined almost 6 percentage points, while 



students from disadvantaged- urban and rural 
communities showed overall gains of 4.3% and 
5%. respectively. The picture shifts somewhat 
at age 13. with blaclcs and disadvantaged- 
urban students showing the only gains of 5.8% 
and 6.1%. respectively Students from 
advantaged-urban and rural communities 
showed declines of over 6 percentage points. 

The most dismal portrait of change by 
groups comes from the 17-year-clds' data, 
where blaclcs were down from 1970-71 on the 
inferential analysis tasic by 12.1 percentage 
points, whites by 9.1 points, students from 
advantaged-urban communities by 10.9 points 
and students from disadvantaged- urban 
communities by 2.4 percentage points. The 
only gains for 17-year-olds were made by 
students from rural communities, with a 4.1% 
increase from 1970-71. 

OonoliMlons 

One of the most telling and immediately 
disturbing conclusions we can draw from these 
findings is that hardly any of the students — 
9- year-olds, 13-year-olds or 17-year-olds 
—showed evidence of having and using a 
systematic approach to the analytic taslcs. 
Even when one talces into account the fact that 
the students were worldng in test-lilce 
conditions with limited time in which to 
respond, the papers are disappointing in this 
regard. Rather than writing open-ended 
analyses that proceed by any one of the many 
possible approaches to analyzing texts (e.g.. 
paragraph by paragraph, focusing on elements 
in the text or following one aspect of meaning 
through the whole text), these students 
wrote quicl(, easy answers focused almost 
exclusively on the content (or action) of the 
text. Even the most soptiisticated papers 
usually laclced systematic approaches to the 
tasl(. Students produced responses that were 
fragmentary, superficial and cryptic; they did 
not go much beyond this kind of response 
to closely analyze the texts or themselves 
as readers with opinions, interpretations 
and judgments. 
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When we look at students' performance on 
multtple-choice inferential questions, we are 
struck by their relatively capable performance 
on these items in relation to their weak 
performance on open-ended items that call 
for analysis and inferencing. Even though 
9- year-olds showed little change on either 
the multiple-choice or open-ended task, the 
change data on 17-year-olds paints this picture 
with the dramatic 10 percentage-point decline 
in the number of students who wrote adequate 
analyses in 1971 as compared with those who 
wrote adequate analyses in 1980. 

Quite clearly, students did fairly well on 
multiple-choice inferential items, but they did 
not go much beyond these ♦asks to the more 
complex, yet fundamental, tasks that asked 
them to explain and substantiate their 
inferences. We are consequently concerned 
that students' successes with quick multiple- 
choice inferential tasks and their failures to 
substantiate their inferences are derived from 
their classroom experiences. Students in all 
age groups might not be getting opportunities 
to engage m the extended discourse, either 
oral or written, that teaches them how to 
explain and substantiate their inferences in 
even the most basic ways. 

We are also dismayed that older students 
did not provide more varying kinds of evidence 
than younger ones. The almost exclusive 
use of content as evidence points to another 
undeveloped aspect of the explanatory 
abilities of these students. We cannot help 
but see this as an extension of their 
classroom experiences, or, rather, their 
lack of experiences with what would take them 
beyond quick, easy answers to thoughtful, 
substantiated explanations of various aspects 
of a text. 

The 13- and 17-year-olds who explained 
their inferences in either barely adequate or 
adequate ways were best at discussing their 
own emotions and worst at discussing theme 
On the other hand, very few 9- year- olds 
(11 5%) explained their emotional reactions 



to a story in adequate ways, although about 
36% of them came up with barely adequate 
explanations. There does, in other words, 
seem to be gradual growth in students' 
abilities to discuss limited kinds of inferences, 
especially personal emotions, as they 
get older. 

There also seems to be evidence that 
emotionally powerful texts like "A Story of a 
Good Dog'' lend themselves to richer, more 
elaborated analyses. Both 13- and 17-year-old 
students incorporated more evidence into 
their reactions to this story than to any other 
selection. It also seems clear that certain kinds 
of texts lend themselves to particular kinds of 
analyses. Some prompt attention to character 
and some, like "A Story of a Good Dog," 
prompt attention to emotions more than 
anything else. 
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CHAPTER 4 

Bvalnating Written 
Works 



In addition to teaching students how 
to make reasoned interpretations of texts, 
another major goal of instruction in reading 
and literature is to develop the students' 
abilities to evaluate written works against 
appropriate criteria. Such criteria cover a wide 
range, including such factors as interest, 
relevance, formal coherence, clarity, 
imagination and social importance. Specific 
criteria will vary with the purpose for reading 
and with the individual reader's personal 
values. Good readers should, however, be 
able to articulate 4he evaluative criteria they 
are using, and they should be able to bring 
these criteria to bear in reasoned ways 
upon texts. 

Two sets of items were developed to 
assess students' evaluative skills. The first set 
focused on the criteria that readers bring to 
bear in their evaluations, asking simply "What 
makes a good story? List three things on the 
lines below." (An alternate version asked what 
makes a good poem.) Each of the items in 
the second set began with a multiple-choice 
question about whether a particular passage 
was a "good story" or a "good poem," 
and followed that up by asking the reader 
to explain what in the passage had led to 
that judgment. 

In answering these questions, students 
have to draw upon many of the same 
comprehension, thinking and writing skills 
required in explaining interpretations of such 
things as mood, character or theme. In 
addition to analyzing texts in support of their 
judgment, however, explaining an evaluation 



forces students to consider the (sometimes 
implicit) evaluative criteria they are using. As 
with other tasks requiring analysis, we would 
expect that the process of elaboration and 
justification would in turn lead readers toward 
a more comprehensive understanding of the 
text; in some cases, it might even lead to a * 
reconsideration of the initial evaluation. 

Resulte: Bvaluatlve Criteria 

The criteria that 9-, 13- and 17-year-olds 
cited when asked what makes a good story or 
poem are summarized in Table 4-1. For 
both stories and poems, the proportion of 
unsuccessful attempts to list specific criteria 
decreased in moving from 9- to 17-year-olds' 
sampies. The proportion of students unable to 
respond to the item at all dropped from 9-12% 
to 2-3%; the proportion listing only one or two 
criteria dropped from 40-42% to 13-17%; and 
the proportion of off-task responses dropped 
from 20-25% to 3-4%. For both stories and 
poems, about half of the students gave at least 
one unelaborated or circular kind of criterion, 
e.g., "the writer," "good writing," "the 
ending," "language" or "good subject." This 
proportion remained relatively constant across 
ages in response to "good story" and rose 
from 40 to 58% between ages 9 and 17 for 
"good poem." Table 4-1 presents the 
percentages of students who cited specific 
kmds of criteria for their evaluations. 
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TaU« 4-1. 

OrttrU OiTta for ■▼Uuati n< StoriM and Voama, Agmu 9, 18 and 17, 1979-00 

Parcent of Students Citing Crittria at Least Once (Out of Three 
Opportunities) 



Blank 

Unelat>orated or circular 
Qenre feature-^unelaborated 
Format only 
Off-ta8l(— illegible 
Content 
Form 

Subjective reaction 
No response to item 

Note. Blai.k = no criteria stated Uneiaborated or circular = superficial criteria Genre feature — unelaborated = 
criteria related to aspects of the genru (the l^md of literature — poem, story, play, etc —being read), like plot Format 
only = critena related to format. Off-task— illegible = unfamble Content - criteria related to content of selection. Form 
- criteria related to aspects of form, like the selection's language or beginning Subiective reaction = personal criteria 



Good Story 




Good Poem 




Age 9 


Age 13 


Age 17 


Age 9 


Age 13 


Age 17 


41 8% 


18 5% 


12 8% 


40 5% 


25 4% 


16 6% 


50 9 


55 5 


53 7 


40 3 


46.9 


57 9 


4 1 


22 7 


31 2 


0 5 


1.7 


2 2 


12,2 


11 0 


4 2 


11 6 


136 


3 1 


25 2 


70 


3 1 


20 1 


12 1 


36 


22 8 


37 1 


43 0 


13 1 


19.9 


31 2 


74 


14 5 


16 3 


45 6 


65 4 


62 4 


23 5 


42 1 


45 6 


18 2 


31 2 


43 9 


93 


2 1 


2 2 


11 7 


5 5 


33 



More specific criteria listed by the students 
assessei were categorized as focusing on 
content, form or subjective reaction. Although 
percentages varied across genres (kinds of 
literature), the proportions o^ students who 
cited each specific category of evaluative 
criteria increased with age, suggesting a 
general inr)provement with age in the ability to 
articulate criteria. 

At age 9, 23% of the students cited 
some aspect of content as an important 
charabteristic of good stories. These 
responses included references to such criteria 
as "mystery stories." "Westerns/' "fantasy, ' 
"suspense" and "drama," as well as ' 
references to theme. Another 24% of the 
students referred to the personal, subjective 
reaction evoked by a story, citing such criteria 
as "interesting.' "funny," "imaginative," 
"suspenseful" or "adventurous." Only 7% of 
the 9-year-olds mentioned any aspect of form, 
including criteria such as "vivid language," 
"suspenseful beginning" and "happy ending." 
(Another 4% mentioned some genre trait, e.g., 
plot or rhyme, without specifying what aspect 
of the genre feature contributed to the 
evaluation.) By age 17, the proportion who 
cited aspects of content increased to 43%, 
subjective reactions to 46% and form to 16%. 



Responses to poems showed the same 
general improvement with age in abilHy to cite 
specific criteria, though the balance among 
criteria differed from that for stories. Even 9- 
year-olds seem to have recognized that form is 
an important aspect of poetry; 46% at age 9 
cited at least one formal criterion for good 
poetry; the percentage increased to 65% at 
age 13 and 62% at age 17. Conversely, only 
13% of the 9-year-olds and 31% of the 17-year- 
olds cited aspects of content in describing 
what makes a good poem. 

The criteria cited by the students were also 
caiegorizad according to whether they referred 
to the work as a whole, required some analysis 
of the text or made some reference to theme. 
Results indicate an overwhelming emphasis 
on the work as a whole at all three ages. For 
good poems, the proportion of analysis rose 
from about 1% at age 9 to about 4% by age 17, 
and the proportion of references to theme 
increased from less than 1% to about 9%. 
Stories prompted less concern with either 
analysis or statement of theme; about 3% of 
the students cited such criteria at any of 
the three ages. 
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m«tiatos axplmlalsj Sraluatioa 



The results reported so far focus upon 
students' abilitieb to formulate evaluative 
criteria; they tell us nothlng^bout their abilities 
to apply these criteria to specific works in 
reasoned ways. This set of abilities was 



assessed in the second series of evaluative 
items, which asked students to explain their 
evaluations of particular texts. Results are 
summarized in Table 4*2. 
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■nvf Liai 


Primarily 




Evaluation 




No 


Of Aeaertiont 


Content 




With 




and 


D«taiit or 




Supportii'g 




cvaiuaiion 


Obaervatlont 


Summary 




Evidence 


Age 9 












story— fable 


20.7% 


23.5% 


53 8% 




2 0% 


Age 13 














7.7 


27.2 


54.2 




i1 0 


folk tale 


14.4 


36.8 


40 6 




79 


Poem— "Mother 








to Son ' 


13.1 


35.9 


46.6 








Avg.= 11.7% 


Avg. = 33.3% 


Avg. = 47.2% 


Avg 


= 7 7% 


Age 17 












Story— fable 


5.1 


36 8 


38.2 




198 


Story— African 








folktale ' 


101 


44.8 


24 6 




20 6 


Poem— "Mother 








to Son" 


6.3 


28 1 


57 5 




8 1 


Poem— metaphor 








death 


15.7 


54 6 


25 6 




4 1 


Story— "One of 








These Days" 


12.2 


48.1 


29 9 




98 




Avg = 9.9% 


Avg. = 42.5% 


Avg =35 2% 


Avg 


- 12 5% 



♦Rows may not total iOO% due to rounding 



At age 9, 21% of the students did not write 
an evaluation that went beyond their initial 
judgment of whether the selection was good or 
bad. By age 17, responses that did not even 
attempt to explain the evaluation fell to ah 
average of 10%, although they continued as 
high as 16% for particular texts. Of those 
responses that did not go beyond the initial 
evaluation, about half were illegible, illiterate 
or blank papers. The other half consisted 
primarily of brief reiterations of the initial 
response — "I just lil^e it"; "It was a good 
story"; "It's the l^ind of story I like." 



Regardless o* passage-type, three-fourths 
of the papers at each age fell into one of two 
rather broad response patterns. The first 
response pattern consisted primarily of a short 
list of rather vague assertions or observations 
about the passages. Though some of these 
responses contained references to the text, 
they Irrjely comprised unsupported 
statements. The second response pattern was 
a summary or synopsis of the story or poem. 
Though the evaluative criteria being used in 
such cases were not stated explicitly, students 
seem to have singled out asnects of content in 
deciding whether a particul story or poem 
was "good." 
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Averaging across specific passages, 
responses relying primarily on lists of vague 
assertions increased from 24% at age 9 to 
33% at 13 and 42% at 17. What is most striking 
and even alarming about these evaluations is 
their almost universal applicability to any 
passage from any book, movie, song or 
magazine. Although evaluations of poems can 
sometimes be differentiated from evaluations 
of stones, many of the responses to quite 
different passages are interchangeable across 
passages and grade levels. This is particularly 
true of the responses at ages 13 and 17, which 
can sometimes be distinguished from those of 
the 9-year-olds* through their use of a more 
sophisticated style and vocabulary. The 
following examples are representative of the 
responses at ages 9. 13 and 17: 

Age 9 

I think the story is good but not super good. 
But it is a dull story. 



It was funny. I thought it was interesting. I 
liked it alot because I have never read a stoiy 
about that. I loved it. 



It was a little eilly with a good ending 



Age 13 

To me it does not seem much like a ^em 
and I don't like that kind of language 



The way she talk and the theme of the poem 
and what it was about 



The story didn't have any suspense llKe a 
good one would have, also, it would have been 
better if the main character was a flrst-pereon 
story teller This would have made the story 
more interesting, 



I tho jght It was very good It served Its 
purpoee The had a lesson in it The s tory w as 
happy at the end 



Age 17 

I decided the story waa a good one on 
the basis that it was attempting to give an 
informative message to the reader 



It was a good story because the main points 

were easily to follo w. The theme was very 

evident, although it did seem a little weak. In 
addition, the story had much action which kept 
the reader interested. 



It is well written, and carries a message. 

Because it kept a very descriptive moving 
pace. It was never at a standstill. 



The story was full of suspense and kept the 
reader in doubt as to the outcome. The author 

uses much description in revealing the 

characters and the s6tting. There is a hidden 
meaning running throughout the story and this 
definitely intrigues the reader. Together with 
the suspense, the extraordinary description, 
and the underlying motive', the author has 
created an interesting story. 

The other major pattern in response to the 
request to explain an evaluation focused on 
the content of the work, through the selection 
of particular details or through a summary or 
synopsis. Such responses were given by 54% 
of the students at age 9, 47% at age 13 and 
35% at age 17 Although students responding 
in this way might be singling out aspects of 
content that particularly interested or 
entertained them, such responses are 
ultimately unsuccessful in that the evaluative 
criteria remained unformulated, and hence, the 
explanation of the evaluation remained implicit 
as well. The following examples illustrate the 
range of responses. As with those that rely 
upon lists of assertions about a text, some of 
these attempts to describe specific content 
were so brief and general that it would be 
difficult to identify the particular passage. 



28 




in 

1 



The ihelff did harm to others but never 
thought he would need help. He was euppoeed 
to be io tough but when it Cwine righ t down to it 
he oouldnt take the pain, IUlq the men He 
killed. 

It waa about a dentist that didn't have much 
respect for the mayor And the reaso n I didn't 
like It was because of the dentist attitude. 



It talks about not giving up on life and even 



though Bhe didn't have a great life she dldn *t 
stop. " 



,1 like the way the mother described her life 
an/l the wsy she encouraged the little boy not to 
quit on his life to Just keep going and there may 
b^ something in life for him. 

The poem is telling her son that life isn 't 
ea^ and you are going to fall on your face a few 
times but don't let it stop you, keep on climbing 
and reaching for the top. 

Only a very small proportion of the 
responses to these items successfully 
addressed the task, including both judgments 
about specific texts and support for these 
judgments. The proportion of successful 
papers increased from 2% at age 9 to 8% at 13 
and 12% at 17. The nature of these successful 
responses varied somewhat depending upon 
the particular passage, and on whether the 
initial evaluation of the passage was negative 
or positive. About half of the successful 
evaluations tended to reach some 
generalization about the purpose of the text, 
as in the illustrations below: 



I think this Is an excellent poem because, 
when I finished reading it. I felt a feeling of 



renewed courage and strength to face outside 
again. It made me think that my life is Just as 
bad as the next person and never to give up 
when I'm feeling discouraged. 

It was the idea of the poem that made me 



choose the answer I did. The poem talks about 
life and that it isn't easy to get to the top. I 
think It describes reality. 



ThU poem is good, because it is filled 

with optomlsm and idealism. The mot her, 
throughout all of her experiences, both bad and" 
good, has been able to "keep her sunny side up." 
She Is encouraging her son to do the same. 
Another feature that gives this poem quall^7 Is 
the style of writing. The grammar in it Infers 
that the mother may not have had an advanced 
education, and writing a poem without that 
kind of education gives the poem a certain deep, 
alfectlonate quality. 

A large proportion of the remainder of the , 
successful attempts at explaining evaluations 
were written by students who did not like the 
passage. Evaluations of the poem "Mother to 
Son," for instance, concentrated on the form of 
the poem, while evaluations of the story "One 
of These Days" criticized the lack of detail in 
the story. Again, examples illustrate the nature 
of these evaluations: 

The poem had words in it like "ain't, kinder, 
goln* " the words aren't good grammar The 
story looked as If it had been written by an 
Ulterate. 



The reason I chose no, is because it w as 
poorly written and somewhat coi.Axslng 



because of all the dump words they use. The 
words yere actually too simple. 



It didn't make any sense. First of all, they 
gave you no clue to who the twenty men were, 

why they died, and how the mayor was 

responsible for their deaths. There were too 
many \inanswered questlc:ie that left holes in 
the story. 

There was no apparent meaning to the story. 

It was ratner simple reading also. The 

narrative did not supply why specifically the 
dentist had distaste for the mayor, or what th e 
purpose was in the dentist mentioning the 



twenty dead men. The story needs to have more 
precise detail, and definitely a theme to it, in 
order to improve the quality of the story. 
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Easmts of Liking the 
Story or Poem 

Each question that requirtd a written 
evaluation of a passage first aslced students 
"Is this a good story?" or 'Is this a good 
poem?" Table 4-3 shows the percentages of 
students responding "yes" or "no" to these 



questions for each passage, as well as the 
percentages falling into each of the four 
evaluative response patterns by their "yes" or 
"no" answers. 
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Several comments can be made about 
students' responses to the initial multiple- 
^choice questions and the relationship between 
their multiple-choice answers and their written 
evaluations. First, from reading the papers, it 
is clear that students generally considered a 
passage to be good if they "liked" it and not 
good if they did not "like" it or did not 
understand it. Second, students generally liked 
the passages. Over three-fourths at each age 



liked the fable, and about two-thirds of the 13- 
year-olds and four-fifths of the 17-year-olds 
liked the poem ''Mother to Son." However, only 
two-thirds of the 17-year-olds liked a rather 
sophisticated poem using metaphor to talk 
about death, and even fewer ^about half) liked 
the story "One of These Days" by Gabriel 
Marquez. 
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Those students who liked a passage 
were generally more likely to write supported 
evaluations than those who did not like the 
passage. This was particularly true for 
passages that most students liked. This might 
be because many of the small number of 
students who answered "no" did so because 
they did not understand the passage and, due 
to this lack of understanding, were unable to 
provide evaluations. For those passages less 
favorably received, students who did not like 
the passages were often less likely to support 
their evaluations than students who liked 
the passages. 

If students liked a passage, generally 
there seemed to be about an equal tendency 
to either provide brief lists of judgments or 
to retell or summarize content. A tendency 
toward one or the other response pattern 
seemed to depend on the passage. If students 
did not like a passage, there was a clear 
tendency toward providing a list of assertions 
and observations. 

Group Porformanee 

When we look at our open-ended 
evaluative groups — race, community, and 
so on —we see that there were no dramatic 
differences between 9-year-old blacks and 
whites nor among students from advantaged- 
urban, disadvantaged- urban and rural 
communities. For 13-year-olds, blacks 
performed 3 to 6% below the nation, while 
students from disadvantaged- urban 
communities performed 1 to 10% below 
students from advantaged-urban communities. 
Students from rural areas performed slightly 
above students from disadvantaged 
communities, but 1.5 to 10 percentage 
points below students from advantaged- 
urban communities. 

Seventeen-year-old black students 
performed 2 to 3 perceniage points below their 
white counterparts on the open-ended 
evaluative tasks, and students from 
disadvantaged-urban communities performed 
3 to 13 percentage points lower than those 
from advantaged communities. 



Females tended to perform better than 
males at all three age levels. Differences were 
slight at age 9 and ranged from 3 to 6.5 points 
at age 13. Differences were more pronounced 
at age 17, ranging from 3 to 13%. There were 
few regional differences in performance on 
any of these questions across the age levels. 
Students whose parents had had more 
education generally performed better on these 
tasks across all age levels. 

Conclusloiis 

Results from the assessment of students' 
abilities to evaluate written works suggest a 
number of conclusions that parallel and 
reinforce those already reported for other 
aspects of writing about literary works. The 
major and overriding finding is that although 
students in each of the age groups assessed 
had JIttle difficulty making judgments about 
what they read, most of them lacked even the 
most rudimentary procedures for explaining 
and defending their judgments through 
references either to the details of the texts or 
to the characteristics of their own responses 
and opinions. Instead, they either used 
stock assertions that can be applied 
indiscriminately across texts or they fell back 
upon restatements of the text without relating 
these summary statements to specific 
evaluative criteria. 

In terms of instruction, each of these 
unsuccessful approaches to evaluative tasks 
can be seen as promising beginnings. The 
formulaic assertions could be treated as a 
framework for elaboration in the contexts of 
specific texts. If a story is "suspenseful," 
students could then be asked to summarize 
the details that contribute to the suspense; if a 
poem has "good rhythm," they could be asked 
to illustrate that rhythm and how it contributes 
to the effectiveness of the poem as a whole. 
Students who began with a summary or 
synopsis need to be shown how to work in the 
other direction — how to take the details that 
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they focused upon and relate them to specific 
evaluative criteria. At an abstract level, the 
responses to the items about "good story*' and 
"good poem" make it clear that most students 
are at least aware of such criteria. What they 
seem to lack is practice in bringing the criteria 
together in systematic ways with details of 
form, content or their own responses. As we 
concluded ^earlier in examining ability to 
explain interpretations of text, students seem 
to lack experience in any sort of extended 
defense of their conclusions, whether through 
discussion that moves beyond brief 
question /answer routines or through writing 
tasks that move beyond short-answer or 
fill-in-the-blank formats. 



Other conclusions from the evaluative 
items can be listed briefly: 

• Responses were strongly influenced by the 
particular text under discussion, as well as 
by general categories of text such as story 
or poem. 

• Students at all three ages assessed were 
. able to cite some evaluative criteria, but 

older students were more specific in the 
criteria they cited and had a wider range of 
criteria upon which they drew. 

• The most popular criteria at all ages focused 
on aspects of the content of a text; similarly, 
attempts to explain judgments about 
particular t^xts focused on content, usually 
in the form of a summary or synopsis. 

• Criteria that imply analysis of a text, as wall 
as those involving generalizations about 
theme or message, where rarely used at 
any age, though they were cited somewhat 
more frequently by the older students. 
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CHAFTERS 

General Responding 

ZAtroductlon 

This final section reports findings about 
achievement in literature from the unstructured 
written responses of 13- and 17-year-olds. 
Students were given a poem or a story they 
had never seen before and asked to "write 
down your thougHts and feelings" or to "write 
a composition" about the poem or story. They 
were on their own with the texts. No other 
directions or questions of any kind structured 
or focused their responses. They had to 
create a meaning for the work, judge the 
appoal of it and then write a response. Such 
an expressed response tells us a great deal 
about readers' experiences with 4ilerature, the 
kinds of classroom discussions and writing 
assignments they have had and4heir thinking 
strategies for expressing their responses 
to literature. 

Altogether we analyzed the written 
responses to four poems ana one story. 
Since 13-year-olds and 17-year-olds wrote 
responses to two of the poems in both 1970-71 
and 1979-80, we can report changes in the 
responses to those selections. Though the 
data reported here come from an analysis of 
the responses to all five works, the examples 
supplied in this chapter are taken from 
"Somebody's Son," a short, somewhat literal 
selection; "Check," a poem by James 
Stephens: and "Into My Heart," a poem by 
A.E. Housman. 

Responses of both age groups at both 
assessment times can be compared because 
the same system was employed for analyzing 
the responses. This system is based on the 
content-analysis scheme used to describe 
responses in the 1970-71 literature 




assessment, a scheme developed by Alan 
Purves and Victoria Rippere (See Elements of 
Writing About a Literary Work, 1968). It 
makes possible a full account of a wide range 
of responses to a work, and it is particularly 
useful for identifying differences in response 
from one work to the next. It relies on the 
following categories of responses: 

Egocentric Responses are not text-based, 
but are text-relevant. Respondent writes a 
letter or poem of his/her own or writes another 
poem (or excerpts) that s/he has memorized. 
Other types of statements categorized here 
are: I never read poetry," "I'm not good with 
poems" or I love the beach — it helps me put 
my mind off things." 

Retelling, f^spondent summarizes rr 
i-etells the poem (or story) or part(s) ut it. This 
can include statements referencing specific 
words or lines. Inaccuracies are disregarded. 

Emotional. Respondent attributes 
emotions or feelings of mood to the text or 
makes a direct statement of emotion. 
Examples would include: "The poem was 
sad," "It's touching," "It had a funny feeling," 
"It was very dramatic," *Mt gave a happy point 
of view" or "The ending makes you feel sorry 
for him." 

Peraonal— global. Respondent gives 
personal reactions to genre and content in a 
global sense. Examples would be statements 
of the following type: "I like poems about 
nature," "I can relate to this poem," "I wish I 
CQuld write poems like this" or "This is not 
my kind of poem." 
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personal— analytic. Respondent gives 
personal reactions to content in an analytic 
sense— identification with characters, 
judgments about actions of characters and 
advice giving, observations about the way 
society should/does worlc. Respondent states, 
for example: "I might have felt the same thing," 
'1t describes my feelings of moving to a 
new state" or "I feel that the poem is right 
by talking about thd real problems of air 
pollution facing us/ 

Evaluation. Respondent judges the 
worth of the work. This also includes such 
statements as: "It is stupid," "I didn't like it," 
"It doesn't make sense." "It is nicely written" 
or "It is imaginative." 

Other works— general. Respondent 
classifies the work as to genre or type and 
compares the poem to other types of works or 
art forms in general, such as: "It's not like a 
poem I ve seen before" or "It is like a myth." 

Other works— specific. Respondent 
compares the poem to a specific work that 
IS mentioned by title, such as: "The Bible 
describes heaven this way." 

Analysis — superficial. Respondent 
mentions superficial characteristics of the 
text. This includes concerns about format, 
for example: The poem doesn't rhyme." "The 
poem seems more like a story" "It doesn't 
give the place and time," "The author uses 
imaginative language or "There is a sense of 
lost beauty in the poem." 

Analysis — elaborated. Respondent 
gives an elaborated or substantive discussion 
of any one of the following special features or 
literary devices plot, character, setting, 
images, sounds, and so on. Included here 
are discussions of plot veracity and 
meaningfulness, such an. 'Even though 
Housman wrote this poem in 1890, it is still 
pertinent and meaningful today" or "Each of 
us has memories of places and people we 
would like to relive, but which time will not 
allow us to ' 



Infsrenclng. Respondent goes beyond the 
text and provides motivations for characters or 
develops action. It includes text-based 
hypotheses of what did happen or predictions 
about what will happen, for example: "The 
author is longing for the home he once had" 
or "A.E. Housman seems to be talking about 
a country that has been badly damaged 
or destroyed." 

Generalization. Respondent derives 
general meanings from the poem, 
, such as: "Inside a man's heart live his 
fondest memories/' 

Though this system has 12 categories — 
a large number of discriminations to make in 
a content-analysis system— raters were able 
to use it after careful training, with high 
percentages of agreement— over S0% on 
each of the five responses to the four poems 
and ond story. Raters coded each written 
response for the appearance of any 1 of these 
12 response modes for the 17-year-olds 
and then coded the entire response for its 
predominant mode. The personal, other works 
and analytic categories were not split into 
subcategories for scoring 13-year-olds' 
responses; consequently there are only 9 
response categories at age 13. Predominant 
mode identifies the major thrust of the essay 
the writer's essential, central concern in 
thinking and writing about the work. 

For assessing reading, thinking and writing 
achievements, such a coding system has four 
special advantages. 

1. It permits detailed description of what 
students do, given a chance to say what 
they want about a work of literature. It 
provides no judgment or evaluation of 
the response. The comprehensive 
description if offers does prdvide 
material, though, for judgments on our 
part about the patterns of response we 
observe. 
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2. It provides information about the 
response preferences or response 
styles of young readers. It shows what 
is common and what is rare in their 
responses and shows how response 
preferences change over time. This 
question of response preference is of 
special importance since recent 
research (Purves, 1973, 1981) has 
suggested that the chief result of literary / 
education in American schools might be / 
to teach not a particular range of / 
reading abilities but a preferred way of 
talking and writing about fictions— / 
movies, poems, stories, novels, pla^s. 

3. This coding system for unstructurjid 
written responses lets us see hoW much 
congruence there is between th6 
declared objectives of literary ir^truction 
in American schools and the responses 
of students to literary works when they 
are on their own without directions from 
the teacher. If schools claim to teach 
students to make thoughtful evaluations 
of what they read using examples from 
the work at hand or from other works to 
elaborate and support their evaluations, 
do they in fact do that when invited to 
say what they want about a work they 
have never seen before? If schools 
claim to encourage a wide range of 
responses to fictions, with each type of 
response developed in a way 
appropriate for that type, will 13- year- 
olds or 17-year-olds as groups display 
such a wide range of responses or do > 
they consistenty shun some forms of 
response in favor of others? Are the 
ones they prefer the ones the schools 
claim to value and teach? 

4. This coding system lets us describe 
both thQ content of the response and 
the probable thinking strategies by 
which the writer achieved the response. 
The written response is a selective part 
of the full response of reading, but it is a 
critically important part. Writing is not 
merely a record of a part of the 



iesponje — it is a way of thinking about 
/ the response. In the act o\ writing, the 
/ readers discover responses. They find 
/ out what they think and feel about the 
/ work as they write. For this reason we 
can say that the written responses in 
this sample— limited as they may be by 
the test situation and the time 
constraints the students were very much 
aware of — reveal the probable thinking 
strategies involved m achieving these 
responses. With this coding scheme, we 
can see readily whether a response 
contains analysis, inferencing, 
generalizations, evaluations, 
comparisons or contrasts, examples, 
personal anecdotes, narrative retelling, 
and so on. 

First we present a complete profile of 
17-year-olds' responses from the 1979-80 
assessment. This section is necessarily 
lengthy because of the importance of 
illustrating findings with examples of students* 
writing. Then, in two briefer sections, the 
performances of 13-year-olds and 17-year-olds 
are compared and changes between 1970-71 
and 1979-80 in responses to poems 
are described. 

How 17-Yaar-blcls Respond 

For the 1979-80 assessment, 17-year-olds 
wrote brief responses to the poem Into My 
Heart" and to the story "Somebody's Son." 
Seventeen-year-olds also responded to two 
other poems; but since these may be used 
again to study changes in responses between 
this current assessment and the next one, they 
can be described only briefly here. 

The story "Somebody's Son" is only 
about 250 words long, the first part a 
characterization of the son and the second 
longer part, his letter to his mother. 
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Read the story below Then winte down 
your thoughts and feelings about the story 
on the lines provided on the next two pages 
We are interested in what you have to say, 
not your aoe'ling and punctuation You will 
have 9 minutes to read the story and write 
your response 

8oA#bodj*8 Son 

He sat, r-iaheu jp on the side of tlie highway, a 
slim, sunbeaten driftwood of a youth He was 
hunched on his strapped together suitcase chin 
on hands, elbows on knees, staring down the road 
Not a oar was in sight Except for him, the dead, 
still Dakota plains were empty 

How 'he was eager to write that letter he had 
kept putting off Somehow, writing it would be 
almost like having com'pany 

He unstrapped his suitcase and f.ahed out a 
small unopened package of stationery from the 
pocket on the underside of the lid Sitting down in 
the gravel of ^he roadside, he closed bhe suitcase 
and used it as a desk 



Dear ICom 

If Dad will permit. I would like to come home 1 
know there's little chance he will T'm not going to 
kid myself I remember he said onoe if I ever ran 
off I might as well keep on going. 

All I can say is that I felt leaving home was 
something I had to do Before even considering 
college, I wanted to find out more about life and 
about me and the best way for us (life and me) to 
live with each other Please tell Dad — and 1 guess 
this'll make him sore all over again --I'm still not 
certain that college is the answer for me 1 think 
I'd like to work for a time and think it over 

You won't be able to reach me by mall, because 
Fm not sure where Til be next But in a few days I 
hope to be passing by our place If there's any 
chance Dad will have me back, please ask him to 
tie a white cloth to the apple tree in the south 
pasture— you know the one. the Qr^mes Golden 
beside the tracks I'll be going by or, the train If 
there's no cloth on the tree I'll Just quietly, and 
without any hard feelings toward Dad — I mean 
that — keep on going 

Love, 
David 



Housman's poem 'Into My Hec^rt" was first 
published in 1890: 

Write a composition in which you discuss 
this poem We are more interested in what you 
have to sfi^y than in how you say it 
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Into my heart an air that kills 
From yon far country blows 
What are those blue remembered hills 
What spires, what farms are those*^ 

That Is the land of lost content, 
I see It shining plain 
The happy highways where I went 
And cannot come again 

AB Housman. 1890 



We will identify the two other poems as 
"Letter" and "Invitation*' (these are not their 
real titles). "Letter" is addressed to a girl by a 
boy who once loved her. He explains why he 
is no longer in love with her and what that 
change will mean for them. The poem 
"Invitation" invites readers to move out into 
the world, to take risks and to be imaginative. 
It is organized around a central metaphor of 
opening a door and looking out. 

Table 5-1 presents the results from a 
content analysis of responses to the four 
selections. The three poems are listed first, 
followed by the story at the bottom. For each 
selection, results are reported in two ways: 

1. As percentages of papers in which the 
response category appeared. If a writer 
used this type of response in even one 
clause or sentence, his or her paper 
was coded for that response. 

2. As percentages of papers for which this 
category identifies the predominant 
response mode. Here the results 
indicate the thrust of the response, its 
thesis or focus, the writer's main 
concern. This is not simply the type of 
response used in the majority of clauses 
or sentences but the writer's basic 
strategy or stance in responding, even if 
that is identified in just one sentence. 
This announcement of basic strategy 
may be followed by several sentences 
carrying out the strategy in a different 
response style. An example would be 
"generalization" followed by several 
sentences of "personal — analytic" 
statements. Another would be 
"evaluation" followed by several 
sentences of "retelling" or "elaborated 
analysis," or a mixture of the two. 
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Tmhlm 1*1. 

1^^*! OaUjorios ♦ 



H9X9b\m EG RT 6M Analytic Qtobal EV olnf tl SipHictflc >JpTl. Elib. tM PEN 



"Into My Heart" 
Ptrcentages of papers in 
wtiich this category 

89^'^ 5 7% 17 6% 4 1% 4.2% 1.0% 15 4% 0.4% 0 1% 4 8% 1 0% 76 2% 4 4% 

Percentages of papers for 
which this category 
kientiflee the 
predominant 

reaponte mode 89 4 3 6 3 1 0 2 1 2 0.3 7 5 0.0 0 0 0 4 0 3 71 1 1 8 

"Letter" 
Percentages of papers in 
which this category 

^•ppeered: 97 3 57 688 193 289 3.3 37 5 29 0.2 74 03 51 8 3 1 

Percentages of papers for 
whi^ this category 
identifiet the 
predominent 

response mode. 97 3 2 6 45 0 3 5 10.6 0.4 15.7 03 00 08 00 17 4 10 

"Irwitation" 
Percenteges of papers m 
which this cetegory 

58 253 3 6 5.2 1.7 39.0 1 4 04 11 5 1 2 1 5 61 8 

Percentages of papers for 
#hlch this category 
identifies the 
predominant 
response mode 

'^Somebody's Son" 
Percentege of papers m 
which this category 
appeered: 

Percentage of papers for 
which this category 
identifies the 
predominant 
respor^e mode- 



' EQ ' egocentric 
RT = retelling 
EM « emotional 
PR Analytic = personal analytic 
PR Global - personal global 
EV » evaluation 

OW General = other works generaf 
OW Specific = o^her works specific 
AN Superf ^ analytic superficial 
AN Elab. » analytic elaborated 
IN s inferencing 
GEN ^ generalization 



98 4 1 3 13 3 0.3 1.1 0.3 23 0 0 0 0 0 1 3 0 1 0.8 56 8 
6 1 6 486 17 6 82.C 0 5 19.2 1 8 4.8 44 05 23 4 1.3 . 

6 11 18 7 1 2 67 2 0.1 6 0 0 1 0.1 0.7 0 1 3 1 0 3 



The first thing to notice in Table 5-1 is that able to say something readable and ratable 

over 97% of the responses to three of the about the selection. Their responses indicate 

selections were ratable. Presented with a that they read the selection and understood it 

literary selection they had never seen before at least well enough to make some appropriate 



and with only a few minutes to develop a written response, 

written response, nearly all 17-year-olds were 
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The second thing we notice in Table 5-1 is 
that the passage overwhelmingly determines 
the type of response. Considering just 
percentages of predominant response modes, 
we see th^t ' Into My Heart" produced 71.1% 
inference responses; "Letter," 45.0% retelling; 
"Invitation," 56.8% generalizing; and 
"Somebody's Son," 67.2% personal analytic. 
No other predominant mode accounts for even 
half as many responses to any one of the 
selections. It seems clear that the particular 
characteristics of the selection— its theme, 
content, language and accessibility — 
determine the initial written response 17-year- 
olds will make. Though these are not results 
for the sarhe individuals responding to all four 
selections, the results do call into question the 
current view that what American students learn 
in their English classes is a particular way of 
responding to literature (Purves, 1981). The 
difference in response mode is more than a 
difference in response to stories and poems: 
the three poems produced quite different 
responses. 

It appears that the main characteristic of 
17-year-olds' wri^en responses to literature is 
flexibility. One work produced quite a different 
response from the next. It also appears that 
most 17-year-olds were capable of at least 
some analysis in terms of their own personal 
experience and of some inforence and 
generalization. 

Though each selection attracted a different 
predominant response mode, the content- 
analysis system does reveal some 
commonalities in the responses to the four 
selections. For example, it is not at all unusual 
to find statements of retelling, evaluation, 
emotion and inferencing together in individual 
responses. Other commonalities are the low 
lnci^3^1ce of egocentric, personal, global (other 
worka). o^^neral (other works), specific, and 
eWc^f superficial .analysis or elaborated 
arjulysis of the selection Two major 
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possibilities of response that 17-year-olds 
characteristically avoided were comparisons to 
other works and developed or elaborated 
analysis of literary features of individual works. 
Although we cannot conclude that 17-year-olds 
were unable to compare works or analyze 
works, it is apparent that when invited to 
respond as they prefer, even the most capable 
of them avoided these responses. 



Response Modes 

The next section of this report focuses on 
preferred response modes— personal analytic, 
retelling, evaluation, generalizing, elaborated 
analysis and inferencing — by looking at 
specific examples of students' responses 
within and across the various literary 
selections. After presenting this detailed 
account of preferred response modes, we turn 
our attention to students' responses to the 
more difficult poem, "Into My Heart." The last 
pages of this section then briefly summarize 
the findings for 17-year-olds. 

As mentioned earlier, the most popular 
predominant response mode for "Somebody's 
Son" was personal analytic— analysis of 
chafv^cters and events in the story from the 
writer s personal perspective, not analysis of 
literary qualities. Within this mode, we see 
writers identifying with characters, judging 
their actions, even giving them advice and 
making observations about the way people 
are or the way the social order works. 
The following four examples of students' 
responses are representative of 17-year-olds' 
responses to "Somebody's Son" within 
this mode. 

In the first example, the student analyzed 
both the father's behavior and David's 
behavior, concluding with what David should 
have done in his letter. 




I f6tl Afct DAVid'fl flither is eltlxer very strict 

or elSfl vary atubbofi x. m^vbs both. W^itlnf ma 
son to <Q to Qolln^a so bad that hs rtriy^g h|m 
awsy from home Is not an Adult ws.y to handle 
a child's releotlon of what a pare nt wants. 
David's father nbmiM have bam ab le to sit down 
with David and digmiased the matter as two 
reasonable and reapo naible adults. 

David's WMT of trying to solve the problem by 
running awav shows that in a wav he too is . 
stubborn. He should have realized that rtmninit 
away doesn't solve someone's problems, it only 
makes them bigtfer f nrt harder to solve. 

He started to get on the right path by 

writing home and aslLing to be able to come 
home ^*in. The onlv thjJig he should have 
dony <^<ff*i rent was to right the letter to his 
father, it would have shown that he oared and 
respy^^^^ hi g father a ni^ hip wiy ^es. but that he 
hlV<? V? ^ft^ e a few decisions himself. 

In the second example, the student 
moved toward generalization in the second 
paragraph, but essentially relied on personal 
reflection and evaluation, as well as on 
advice-giving. 

T ^^^n )^ this is kind of a sad story, very true 
to life. Th e bov feels the wav I do. wanting to 
And out about life and himself. 

The only way to find out what's out there 
ia to go out there yourself. One has to be an 
adventurer to find out about life's many facets. 
People who don't go out and explore life are ,1UBt 
living in little boxes, always knowing tomorrow 
will be the a^me fc ^dav. It's easy to die of 
horedom the wav most people liv e: alwava the 

same thing. Parents don't understand this 

need to explore. 



The third response indicates strong 
identification with David and predicts a bad 
end for him. 



I think this is a sad story. The dad makes me 
mad. He should let his son leave flf come back if 
his son feels a need to go out in the world y 
And himself.^ then decide to come home. If he let 
his son come back, he (Davids would feel more . 
secure and mav-be go to college if he had a little 
adult support behind him, respeciallv his father 
— beceuse of the relationship a father y son 
usually havey How David is lust f oing to go on 
running and for th^ rest of hi g life he will 
probably be a tramp, with no food, or money. 
and finding smaU. dirty lobs thut d on't oav 
good. I don't see how the father can Uve with 
himself knowing what he m rtniT^ff to hifl fl^>n 
not ifttttntf him come back home. 

The fourth response judges David harshly 
and conjectures about what might have 
happened had David not run away. 

I feel that he was wrong to run awav. Nfa 
matter what his problem is he woiUd have been 
able U> work it out. By running awav he hurt 
hie parents and his father might not let him 
come back. You ahouii^n 't run away from vour 
problems you should try to face them and work 
them out. If he had staved and talked about how 
he feels about goin_g^to college or not being sure 

that he w ^ntj^ to f ^s father might have 

understood. 



These four responses represent very well 
the full range of personal analytic responses in 
our sample. They illustrate the overwhelming 
attraction to personal analysis in respond- 
ing to "Somebody's Son"; 672% of the 
responses were of this type. Seventeen- year- 
olds found David's plight realistic. They liked 
the story, and they found it close enough to 
their own experiences and concerns to be able 
to identify strongly with the situation and to 
have something to say about it. There may 
be some evaluating or generalizing and 
occasional short sections of retelling in these 
responses, but there is rarely any hint of 
analyzing any of the literary features of the 
story. A few students mentioned the similarity 
to the song "Tie a Yellow Ribbon Round the 
Old Oak Tree." Two of the examples (2 and 3) 
began with a brief statement of evaluation, and 
two (1 and 4) moved directly to personal 
analysis. The writers seem to have treated the 
events as real events, as though the selection 
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were a case atudy report rather than a short 
story. They were quick to judge and to advise. 
They seem to have felt they must propose a 
solution to the problem or to conjecture about 
how it might have turned out differently. 

Because these responses are unique to 
this story, our comments should not be taken 
as conclusions about how 17-year-olds 
generally responded to literature. Indeed, one 
of our major findings is that these same writers 
would probably respond quite differently to a 
different kind of story or poem. Although we 
cannot claim that 17- year-olds predictably 
analyzed what they read in personal terms, it 
is apparent that they did i) readily to stories 
with people and events close to their own 
experience. 

When we look behind the content of these 
personal analytic responses to their manner of 
development, to the thinking and writing 
strategies apparent in them, it is obvious that 
the responses do not go very deep. There 
was very little expression of tentativeness. 
ambiguity, exploration of various implications, 
judgment or conclusion. The typical sentence 
was an unqualified assertion. Though all of 
these writers know firsthand the complexities 
in family dramas, particularly the tension 
between parental authority and adolescent 
freedom, the papers revealed very little of this 
complexity. There was hardly any exploration 
or conjecture about motives: either David is to 
blame or the father is to blame or both are to 
blame and that is about it. The characters* 
motives are accounted for simply in terms of 
stuboornness, strictness, running away 
and rejection. 

In terms of writing strategies, the writers 
characteristically relied on abstraction. Paper 
4 IS a good example of this, it is a series of 
assertions all at the same level of generality. 
The three sentences of the first paragraph in 
paper 1 are also typical, evaluation, judgment, 
advice, all at a high level of abstraction. Only 
occasionally is there a move downward to 
specifics, either from the story or from the 



writer's experience. One example of a move to 
specifics is toward the end of paper 2. where 
the writer specifies partici'i'^r features of 
David's bleak future: "no food, or money, and 
finding small, dirty jobs that don't pay good." 
Another is the sustained development eft the 
"need to explore" in the second paragraph of 
paper 2: the mention of "little boxes ' and 
dying of boredom. 

But these are exceptions. Even though 
the papers are coherent, they are not 
well developed. The wide range of writing 
strategies available to these writers^ 
definition, example, analysis, anecdote, fact, 
detail— were hardly used. And while it is 
true that time was limited and the writing 
task mentions only "thoughts and feelings," 
we expected a fuller range of thinking and 
writing strategies. 

Retelling 

The next most common kind of response to 
"Somebody's Son" was retelling, the simple 
recounting of what happened in the story, 
with little, if any, ref irencing, analysis or 
generalization. This predominant response 
mode accounted for 18.7% of the responses. 
The following examples illustrate this kind of 
response: # 

This boy was at the point where he felt lllca 
leavlrrf home. He left and was altting on the 
side of a highway In the DakoU Plains. He waa 
somewhat lonely. He was not sure If leaving 
home was the right thing to do. so he wrote a 
letter tc h is parents, his mother preferably. He 
.wanted t o qif><n mi t if his father would except 
him bao k home. His father once said that if his 
son ever left he would not be able to come back. 
With hin fl on knowing thia hn At.i u wanted to 
come baok ov^plnnkin g what hia father said. He 

wanted to mjt^a bac k beoauae he was not 

certain what he want ed to do with his life vet 

This is a story that is a^o ut a kid whose 

dad and hiia were arfimlng abn ut oolle^fe and 
he wasn't sure he wa nted to go to oQiis^e 'so 
he split and found out what life r eally la about 
and is sorry he left and wants to oome back but 
his father said once t hat if he took off not tn 
come back so he does n't know if he oan oome 
back home. 
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These retelling responses were often 
complete (they surveyed all the main events of 
the narrative) and orderly (they followed the 
chronology of the narrative), but they seem a 
limited response to "Somebody's Son, " even 
to the simple request to write about "thoughts 
and feelings. " The retellings were not * 
illustrations for inferences, evaluations or 
generalizations; they were simply retellings. 
For nearly 20% of the 17-year-olds responding 
to this story, a simple retelling was the 
predominant feature of their response. And 
except for the low incidence of retelling in 
response to "Into My Heart" (3.1%), retelling 
was always a popular response mode (as we 
see in Table 5-1)» with 45.0% to "Letter" and 
13.3% to "Invitation?" While there was notable 
variation in retelling among these three current 
works ("Into My Heart,"" "Letter,"" "Invitation""), 
it appears that, on the a /erage, retailing might 
account for up to one-fourth of 17-year-olds" 
written responses to unfamiliar works of 
literature. 

Evaluation 

While there were only a few evaluative 
responses (6.0%) to "Somebody s Son, " other 
selections in the assessment attracted a 
higher proportion of evaluations (see Table 
5-1) from 17-year-olds. They often appeared in 
* a sentence or two embedded in responses that 
are predominantly in another mode. As we 
have seen earlier, it is not unusual 4ar a 
response to open with a brief statement of 
evaluation and then move on to other matters. 

The first two of four representative 
examples of evaluative responses indicate the 
characteristic brevity of evaluation. Instead of 
moving on from general evaluation to some 
Other kinds of statements about the story, 
these writers remained where they began. The 
responses have a repetitive or additive quality. 



— I don't reaUY IDre tha Btora It la not vry 
eXQimu. it Ifl sort cf borlntf. I'm not a^vinf I 
hate It but U ^uat lan't my Wntj ptorv. VTimta 

I have to BBY Abou t it. Bxcapt thmt It ifl or 
oould be true baofcu ae there ara probAUv a iot 

of kidfl in real life that are like the ana ir 

the fltory. 

I liked the atorv. but to me. because I heard 

the aonff fLrat the a torv seemed to be a take off 
gpom tha " Tie a Yellow Ribbon Vound the oY Oak 
Tree." I aIaq think it ahould bava a d^ f farant 
titia Ar\t\ A andi^g because it leaves you curious 
aa to whftthar or n ot Davlda^ Dad let him come 
hamft, 

The third example, by contrast, was 
coherent and rather fully developed, rejecting 
the story for a great many difiarent reasons. 

This atorv, while emotionally touching. 

relies heayily on atoc k responae. It is guilty of 
£flatimentalitY by going into exoeaaj ye detail 
(e^ 'gxcapt for him ..amnty" "a l .n. sunbeaten 
ilriltwQOd"^ and by the uaintf of Tc t^rda tb 
evoke emotion (b^ "huncbad/' ^' aunbaatan 
IriftwQod"^. Very littla la ; old about fthAPacter 

or theme, and thfl fltory In i 'adominitntiy plot, 

Tha plot itaalf la tha "Tift > <Uow Rlhhon" 

fltflreotypft. There are nn t l. ^ .inuaiona ^nd 

few metaphors ara uaad T lerma nf Ut^arary 
critiGlam the atnry la poor^ a van jndgad aa 

escape lltaratnra Whlla g ^ for anma 

dlvaralonary readings thi g atory la not a yary 
good nna 

The fourth response was one of the most 
impressive and imaginative in the sample. 
There were very few responses like this. The 
writer skillfully impLed dislike for the story in a 
list of quotations from imaginary reviews. 

Sentimentllv devastating. A true tear ierker 

rNew York Tlmaa-> 

Kurt Vonn egut Jr's. best eyerlll (LA. Post) 

— A truely a mazing drama of an American 
BXPerience. Upliftinf f or the whole famUy 
Quxisla''a^-ll 

The Ac. ♦^vaalailtn ftgH experlman t is as 

fUinHInf aa a daoTyMbo nauolear Aftld 

achamg.tln (P opular aolanoa;^ 

1 hone there ain't a mo th. (Paaalinist^ 

1 belieye in mlpadaa. /- Chrlatlan flQcie^y;> 

So good I had to raad It afain V nnaatlyt 

Wlna miniitaa la too long. If you don^t mind 

au^faatlona Mn T dr> p*t naad mora apaoa. 
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Though we did not rate the evaluative 
responses for extent of elaboration or 
development, our impression from reading a 
great many of them is that atx>ut half were 
undeveloped. Papers 1 and 2 are examples. 
Again, using quite rough averages across all 
four works, as we did in the discussion of 
retelling earlier, we could reasonably conclude 
that 25-30% of the 17-year-olds chose to 
respond with either simple plot summary 
(retelling) or with unsupported judgment 
(evaluation). For more information about the 
criteria students used in their evaluations 
and the substance of their evaluative remarks, 
see Chapter 4. 

Generalizing 

Except in responses to "Invitation," where 
it accounted for 56.8% of the r<)soonses, there 
was very little sustained generalization in 17- 
year-olds' responses to the texts. In this sort 
of response, the writer is concerned with 
what the work might mean in a general way. 
The response is noticeably less personal 
than in papers classed as personal analytic. 
The following are two examples of this 
response strategy: 

• 

I feel that the story was meant to teach 

tee aagers that they will learn about life when 
Kie time comes They should not run away and 
go out alone until they fully can undfli^atand 
and accept the responsibilities thev muat meet 
to live on their own In this case David decided 
to go out on his own without his parent's 
approval, before It was his tlm A to do this. Nnw 
he wants to go back because the challenge is to 
great for him. This paragraph also shows that 
parents should not force a child to dn thinga 
they don't want to> as It appears that David la 
scared of his father because he does nnt want 
to go to college ^ . 



I thXak that this story la very touohinn. It 



showB eomethlim. I feel, is rarelv seen, aomeone 
wUUni to admit they may have made a miatake. 
Moat people woxildn't do that. \ 

I also feel that this story gives an example 
that you can't always have things like you want 

them to be. In other words, things are n't 

always as you plan them, they mabe l>etter - 

or worse. 

David realizes he may have been a bit wrong 
ftP<' y}f yiTnitfl that. I think thin took a ^frealf 
deal of oourage. He knows that h e mav not 

be able to hgTHf f ff^iP- b^t he aeema to be 

willing to accept it if h ^ fl^^nr^ff^ 

Notice that in both of these examples 
retelling was used to develop the major 
generalizations that are the focus of each 
response. However, very few generalization 
responses displayed thinking and writing 
strategies beyond simple retelling. Data in 
Table 5-1 for "Invitation" illustrate this claim 
very nicely. Invitation" was the only work 
that invited a high percentage (56.8%) of 
generalization as a predominant response 
mode (second row of figures under 
"Invitation*'). Looking at the incidence of all 
response strategies in individual papers (first 
row of figures under "Invitation*'), we see that 
evaluative responses appeared in 39.0% of 
the papers, retelling in 25.3% and superficial 
analysis in 11.5%. The appearance of 
strategies other than retelling and superficial 
analysis that would indicate development of 
generalizations— personal analytic, elaborated 
analysis or other works (specific) — was so 
sparse as to be hardly worth mentioning. 
Again, as was the case with personal analytic 
responses above, the responses were weak 
and poorly developed. 

Elaborated Analysis 

Finally, the respo'^ses to "Somebody's 
Son" illustrate one type of response that 17- 
year-olds consistently avoided: elaborated 
analysis of features of the story as a literary 
text. Both the following examples are excerpts 
from slightly longer responses; and while 
they are refreshing and precise, they are, 
nevertheless, rare exceptions among the 
responses 17-year-olds wrote. 
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I Ilka thft nymbnUfiin uaad At t he flMt of 
tha atnrv A mn^w^y taaniLtfar r^Allv oan ba 



QQXnpared to > PleoB of drlftwQ>^ d. Ha luat fata 
"WMhad" from hara t o thare with the now Qf 
^i\ntn TTlft mv tiTlftti David u da^orlbad mttlnff 
there on the elde of tha road t ha reader o^n 
PlQtura It aAfllly 



The OPeninit of this pasaatfe la very 



descrlpttve and giv es a olear impreflalon of 
what David muet have looked Ilka on the 
Dakota hltfhwav. I thin k that driftwood la an 
appropriate oholoe of worda. for David haa left 
home and la driftin g from town to town. The 
Plalna heln^ amntv amt hla mno d. He la ampty. 
without raaourftaa A nd ready to ^o hnma 

We now turn our attention to a small 
selection of responses tc the A.E. Housman 
poem "Into My Heart." This poem is quite 
different as a literary text from "Somebody's 
Son/' It is compact and metaphorical, not 
discursive or literal. It deals with a theme 
closer to the experience of aging adults — 
acute longing for remembered scenes and 
events— than to adolescents. It is allusive and 
indirect: the reader must understand "air that 
kills'* and "lost content." 

Write a compoBitioh in which you discuss 
this poem We are mor^ interested in what you 
have to say than in how you say it 



Into my heart an air that kills 
From yon far country blows 
What are those blue remembered hills 
What spires, what farms are those*? 

That is the land of lost content, 
I see It shining plain 
The happy highways where I weni 
And cannot come again 



AE Housman, 1890 



Nearly all of the ratable responses to 
"Into My Heart" were inferencing (71.1%). The 
difficulty of thb poem seemed to require 
readers immediately to adopt an inferencing 
strategy. Since the experience of the speaker 
in the poem was not close to their own 
experience, they rarely used persona! analytic 
(the favored response mode to "Somebody's 
Son") or generalization. 
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These three following examples are 
representative. Two interpret the poem as a 
contrast of industrialization and urbanization 
with country life, one as remembered war 
scenes. Many students interpreted "air that 
kills" to mean air pollution. 

1 don't reAllv i:nrt»rflt and what it la about. 

How thftt I rftftd It AqtLin T think i can try to 
an&lYZft It though. It's mhaut h ow lndufltrlii, ]tzfld 

thfl nnuntrv waa ^f|t.t1ng TT^ w hl^hwAVfl 

rflPlftOftd dipt vntLAa and hnw bitf cltlefl replaced 
littlft tnwnii Thifl par*inn, A TP. HmiHmRn , RftflTnart 
flad that thla wan happening I'm nnt rftally aurft 
if thifl ifl nnrrAftt hnt T maria a try at It. 



The poem la talking about a person that has 

luflt went thrnutfh Aome kind o f war or battle in 
their homalaT^d and that per qon Ifl deanrlblng 
what the land and flurroun dings looked like 

after tha war was o ver. Everything baa 

Ohantfed. nomehow dlfferi^n^. Thft ninft 

mirroiindlntffl «.rfl loat forev er. The war will 
Alwava be remembe red. It could have been a 
war between two c ountrleft 



InihlB poem the Author la re miniscing 



about the past, qe r emembers the farm country 
and how he was happy and nonte nt there He 
remembers all of t his, but knows he cannot go 

back to it. I think he Is referring to how 

country land was ohan^ei int o a city and the 
air that ki lls might mean air pollution 



These responses indicate a sensitivity 
to the tone of sadness and regret in the 
poem, and they offer a consistent, focused 
interpretation, However, the writing strategies 
selected to develop the interpretation were 
quite limited. Very few writers used any forms 
of analysis (see Table 5-1). Almost none 
mentioned other works. The papers were 
characteristically a senes of inferencing 
assertions, all at the same level of abstraction 

What does all this reveal, then, about the 
initial written responses to literature of 17-year- 
olds? 

• Even with severe time constraints, they 
were able to formulate m writing a coherent, 
readable, brief response to a written work 

• The focus of their response, its predominant 
response mode, was heavily dependent on 
the work of literature, a discursive work 
close to students' own expenence produced 
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personal analysis, while a metaphoric work 
dealing with unfamiliar themes produced 
inferencing. 



• At least one*quarter of the responses 
were limited to retelling the story or to 
unsupported evaluation. 



• Favored response modes were personal 
analysis, generalization, inferenSing 
and retelling. 

• Disfavored response modes were 
comparisons to other works and literary 
analysis. 

• Responses were not developed or 
elaborated and displayed a limited range of 
thinking and writing strategies. Seventeen- 
year-olds were able to analyze in terms of 
their personal experience, to interpret and to 
generalize; but they seldom developed or 
supported these higher-level responses. 



We now turn to a comparison of responses 
of 13- and 17-year-olds. 

Differences in Hesponses 
of IS* and 17*Tear«01d8 

Table 5-2 summarizes the results of the 
content analysis of 13- and 17-year-olds' 
responses to the two unreleased poems 
"Letter" and ' Invitation" and to the released 
story "Somebody's Son." Data for 17-year-olds 
in this table are the same as in Table 5-1. 



Table 8-a. 



a«sponM /Writing Strategies in 15- and 17*Tear-01da* Written Besponses in 1979*60 to Two Poems and One 
Story 



Works AgM 



"Letter" 
Percentages of papers 
in which this category 1 3 
appeared 1 7 

^Percentages of papers 
for which this category 
identifies the predomi* 1 3 
nant response mode 1 7 

"Invitation" 
Percentages of papers 
m which this 13 
category appeared 17 

Percentages of papers 
for which this category 
identifies the predomi- 13 
nant response mode 17 

"Somebody's $on" 
Percentages of papers 
in which this 13 
category appeared 1 7 

Percentages of papers 
for which this category 
identifies the predomi* 13 
nant response mode 17 



Ratabto EQ 



RT 



EM 



Categories* 

PR PR OW OW AN AN 

Analytic Global EV Genaral Spacific Suparf . Elab. 



IN 



GEN 



95 8% 9 6% 56 0% 25 9% 28 7% — 510% 5 6% — 115% — 36.9% 2.4% 
97 3 5 7 68 8 19 3 28 9 3 3% 37 5 2 9 0 2% 7 4 0 3% 518 3.1 



95 8 3 9 30.4 68 10 4 — 32 706 — 
97 3 26 45 0 35 10 6 04 15 7 03 00 



97 2 80 40 8 10 9 62 — 62 5 42 — 

98 4 58 25 3 36 52 17 39 0 14 04 



97 2 22 22 6 18 19 — 43 5 03 — 

98 4 13 13 3 03 11 03 23 0 00 00 



97 3 5 0 42 4 21 0 74 0 — 22 7 4 1 — 

98 6 1 6 48 6 17 6 82 0 0 5 19 2 1 8 4 8 



1 3 — 88 08 
08 00 174 1.0 



169 — 
115 12 



3 1 — 
13 0 1 



4 7 — 
4 4 0.5 



0 0 26 0 
15 61.8 



00 21 8 
0 8 56.8 



29 1 12 
23 4 1.3 



97 339 16 8 35 56 8 — 9903 — 09 - 51 02 

98 6 11 18 7 1 2 67 2 0 1 6 0 0 1 0 1 0 7 0 1 3 1 03 



*EG egccentnc 
RT retelling 
EM ^ emotional 
PR Analytic ^ personal analytic 
PR Global - personal global 
EV - evaluation 



OW General other works general 
OW Specific - other works specific 
AN Superf - analytic superficial 
AN Elab ~ analytic elaborated 
IN mferencmg 
GEN generalization 
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Considering first any age differences in 
predominant response mode (the third and 
fourth rows of figures under each work), it 
appears that for the two poems a different 
mode predominated for each age group: for 
"Letter," 17-year-olds preferred retelling while 
13-year-olds preferred evaluation with retelling 
close behind; for "Invitation." 17-year-olds 
preferred generalizing while 13-year-olds 
preferred evaluating. By far the most common 
response among 13-year-olds was evaluation. 
Comments of evaluation were more likely 
to appear in all their papers, whatever the 
predominant response mode. For "Somebody's 
Son," both 17-year-olds and 13-year-olds 
preferred personal analytic responses. 

In general, the responses of 13-year-olds 
were much more likjiy to contain statements 
of evaluation. They also consistently included 
a slightly higher percentage of egocentric 
comments, though it remains a small 
proportion of the total. Their responses to the 
two poems were much less likely to contain 
generalizing and inferencing. 

Though the response patterns for the two 
age groups '\re not at all sharply different, 
there is a dual decrease in evaluative 
responses across the high school years and a 
slight increase in inferencing and generalizing 
responses. We would have predicted such a 
shift. ^Except for these small differences 
observable in Table 5-2, our general 
impression from reading a great many 
responses from both age groups is that the 
17-year-olds are not much more capable 
than the 13-year-olds of developing and 
elaborating their responses^ Indeed, Table 5-2 
indicatec that the response of a 17-year-old is 
no more likely to contain analysis (personal, 
superficial) or reference to other works than 
the response of a 13- year- old. 



Results: Dlfferancss In 
Responses 

Between 1970-71 and 1979-80 

Was there any change in 13- and 17-year- 
olds' response strategies during the decade 
of the seventies? In both 1970 and 1979, 
13-year-olds wrote responses to the poem 
"Check" by James Stephens, and 17-year-olds 
wrote responses to "Into My Heart." The latter 
is reproduced earlier in this chapter. "Check " 
is reproduced below: 

You are going to be asked to write a 
composition about & poem. I will read the ppem 
to you as you read it to yourself When we have 
read the poem carefully, write a composition 
in which you discuss the poem. We are more 
interested in what you have to say than in how 
well you say it. Put the title of the poem at the 
top of the next page. 



Cheek 

The Night was creeping on the ground! 
She crept and did not make a sound, 

Until she reached the tree- And then 
She covered It, and stole again 

Along the grass beside the wall! 
— I heard the rustling of her shawl 

As she threw blackness everywhere 
Along th« sky, the ground, the air. 

And In the room where I was hid) 
But, no matter what she did 

To everything that was without 
She could not put my candle out! 

So I stared at the Night And she 
Stared back solemnly at mel 

James Stephens 
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Table 5-3 displays the results of the 
content analysis. Between 1970 and 1979, 
for 13-year-olds, there was a marked drop in 
the number of papers with retelling as the 
predominant response mode and a substantial 
increase in papers with evaluation as the 
predominant mode. There was also an 
increase in superficial analysis. The main 
finding for 13-year-olds is simply less retelling 
and more evaluation. 
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How good are these new evaluations? We 
cannot be certain, but the data in Chapter 4 
and m Table 5-3 do not hint at any increase in 
percentages of papers with strategies (the first 
two rows under ' Check") we would expect 
to find in evaluations that were justified and 



supported. The drop in retelling is not offset 
by anything that suggests noticeable 
improvements in other kinds of achievement — 
that IS, by more elaborate and mature thinking 
and writing strategies. 



Ohanfts in R#spons«/Writinf 8trat«Ci«s of 18- and 17-Te«r*0ias Between 1970-71 and 1979-80 



Works 



Tears 



Categoriee * 



Ratable EG 



RT 



EM 



PR PR 
Analytic Global EV 



OW OW AN AN 
General Specific Superf . Elab. IN 



GEN 



Check" (age 13) 
Percentages of papers 
in which this category 
appeared 

Percentages of papers 
for which this category 
identifies the predomi* 
ns'^t response mode 

"Into My Heart" (age 17) 
Percentages of papers 
in which this category 
appeared 

Percentages of papers 
for which this category 
identifies the predomi- 
nant response mode 



197U-71 88 9% 
1979-80 90 2 



1970-71 88 9 
1979-80 90 7 



1970-71 91 7 
1979-80 89 4 



1970-71 91 7 
1979-80 89 4 



4 1% 
6 3 



2 3 
39 



3.1 
5 7 



1 9 
3 6 



76 8% 
69 5 



66 2 
56 7 



25 5 
176 



3 3 
3 1 



7 9% 
11 8 



08 

1 'S 



2 2 
4 1 



00 
02 



3 3% 

4 9 



05 
08 



5 8 
4 2 



1 8 

1 2 



9% 
35 1 



0 8°/ 

1 0 



02 
03 



15 6 
21 1 



69 
154 



2 5 
7 5 



1 5% 

2 6 



0 1 

C 4 



04 
04 



00 
00 



0 9% 
0 1 



02 
O'^ 



15 3°/ 
21 5 



2 6 
53 



36 
48 



03 
04 



0 1% 1 7% 
00 11 



00 
GO 



08 
04 



2 1% 
1 0 



0 5 
03 



87 0 4 7 
76 2 4 4 



79 9 
71 1 



1 1 
1 8 



*EG - egocentric 
RT ^ retelling 
EM - emotional 
PR Analytic - pei'sonal analytic 
PR Global ^ personal global 
EV ^ evaluation 



OW General other works genera' 
OW Specific = other works specific 
AN Superf analytic superficial 
AN Elab analytic elaborated 
IN ~- mferencing 
GEN generalization 



For 17-year-olds during the 1970s, there 
was a large decrease in inferencmg 
'•esponses. The only offsetting increases were 
in the egocentric and evaluative categories. 
The former nearly doubled, though it remained 
small as a proportion of total responses. The 
latter tripled. Again, Table 5-3 offers no 
evidence that the increase in evaluation 
represents an increase in elaborated 
evaluations rather than the simple assertive 
ones we saw previously for "Somebody's Son" 
(evaluation papers 1 and 2). In fact, there were 



slight decreases in the percentages of papers 
with any mention of analysis (personal or 
elaborated) or of other works (other works 
specific). 

What IS the meaning of 17-year-olds' drift 
away from inferencmg toward evaluation? We 
can only conclude that they— and 13-year-olds 
as well — are more likely at the end of 
the decade than at the beginning to make 
judgments rather than to interpret and analyze 
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Average Percentages of 
Scorers' Agreement for 
1979-80 Open-Bnded 
Scoring^ by Age Qtooups 



Somebody's Son" 

Somebody's Son " (essay) 

A Story of a Good Dog" 
"One of These Days" 

Old Dog ' 

Mother to Son " 

I was you" 
Good story 
Good poem 

Check' 

Into My Heart" 
As the Cat" 

The Closing of the Rodeo" 



Av«rag« % of Agr««m«nt 



Ag«9 


Ag«13 


Ao«17 




95.0 


95.0 




95.0 


93.0 




95.0 


94.0 






94.0 


98.0 


91.0 






91 0 


91.0 






93 0 


98.0 


93.0 


940 


98 0 


96.0 


94.0 




965 








96.5 


96 5 








940 


94.0 
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APPENDIX C 

Bzercises and Scoring 
CKildes 



1. Somebody's Son Exercise: 
Format 1 

Read the story below and then answer the questions 
on the next two pages 

Somebody's Son 

He sat. washed up on the side of the highway, a 
slim, sunbeaten driftwood of a youth. He was hunched 
on his strapped together suitcase, chin on hands, 
elbows on knees, staring down the road. Not a car was 
in sight Except for him. the dead, still Dakota plains 
were empty. 

Now he was eager to write that letter he had kept 
putting off. Somehow, writing it would be almost like 
having company. 

He unstrapped his suitcase and fished out a small 
unopened package of stationery from the pocket on 
the underside of the lid Sitting down in the gravel of 
the roadside, he closed the suitcase and used it as a ' 
desk. 

Dear Mom, 

If Dad will permit, I would like to come home. I 
know there's little chance he wilf I'm not going to kid 
myself I remember he said once if I ever ran off I 
might as well keep in going 

All I can say is ^tat I felt leaving home was 
something I had to do. Before even considenng 
college, I wanted to find out more at>out life and about 
me and the best way for us (life and me) to live with 
each other. Please tell Dad and I guess thisH make 
him sore all over again — I'm still not certain that 
college is the answer for me I think I'd like to work for 
a time and think it over 

You won't be able to reach me by mail, because I'm 
not sure where I'll be next But m a few days I hope to 
be passing by our place If there's any chance Dad will 
have me back, please ask him to tie a white cloth to 
the apple tree in the south pasture— you know the 
one, the Grimes Golden beside the tracks I'll be going 
by on the tram If there's no cloth on the tree I'll |ust 
quietly, and without any hard feelings toward Dad — t 
mean that— keep on going 

Love. 
David 



□ 
□ 
□ 
B. 
■ 
□ 
□ 
□ 
□ 
C. 



□ 
□ 
□ 
□ 



A. Why did David write the letter? 
□ To tell his mother that he had decided to go to 
college 

To get his father's approval to return home 
To ask his parents to send him money 
To let his parents know he was leaving home 
I don't krww. 

When does David hope to be riding bv home'^ 
In a few days 
In two weeks 
Next year 
Never 

I don't know 

What kind of person does David think his father 

IS? 

Stubborn and untxending 
Weak ^d uncertain 
Easygoing and carefree 
Fair and Understanding 
I don't know. 

D. Think about the story again What kind of person is 
David? Describe David in a few words on the line below. 

E. What was it about the story that led you to describe 
David the way you did in Question D? Write your answer 
on the lines below 



Scoring Guide — Responding to 
Literature 

Explaining Responses to Literature 

Inferencing — Character 
"Somebody's Son** 
H-401000-B1B-2, 3 
Age 13, Package 10, Exercise 3 
Age 17, Package 10, Exercise 6 

General Scoring Rationale: Since the responding to 
literature obiective was formulated to address "deliberate, 
conscious kinds of interpretation," a successful response 
not only should identify a character trait appropnate to 
David but also should explain the given interpretation by 
relating it to the text Evidence can be given by citing 
specific events in the text or special aspects of the 
construction of the text 
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NOTE Rating should, in general, be done by taking 
the entire student response into consideration without 
regard to what is actually written in the space provided for 
the first part and for the second part of the response. If 
reasons (substantial) are given m the first par*, they are 
valid The same is true for character traits identified m the 
second part Caution should, however, be used, as this 
often leads to a tendency to rewrite the responses. 
Categorizations should reflect, as closely possible, 
what respondents actually wrote 

Scoring Guide Categories: 

I. First Categorization — Identification and 
substantiation of character traits This takes into account 
both ooen-ended parts 

1 ^ Unabie to identify character traits. 
Respondents do not do the task. They refer to the text, 
but do not answer even the first question 

A. Only an opinion about the action of the character 
IS offered, such as David shouldn't have left home " 

B. Some material is quoted from the text with no 
clear identification of character (including quoting title). 

C. The identification and substantiation of character 
seem unrelated to the text 

D. An observation about the story is made, for 
example The title is nn^ieading " 

2 Character trait identified without 
substantiation. Respondents name something but 
cannot go on They identify a character trait(s) but do not 
substantiate the choice(s) with evidence from the text 
Responses tend to provide 1) circular evidence, 2) a 
copy or Close paraphrase of the text, 3) vague reasons, or 
4) only a subjective reaction as substantiation 

3 Character trait identified and substantiated 
with minimai evidence. Respondents identify a 
character trait(s) and substantiate their choice(s) with 
only one reason or piece of evidence related to the text 

A. Reason can be directly related to the text, for 

example Nice he wants to come home ' 
B Reason can be mferred from the text 
C Reason can be inaccurate if it is related to the 

text for example Smart since he finally decided to go 

to coitege 

D Reason can be based on personal experience that 
IS related to the text 

E Reason can be unusual such as Sunburned 
from Sitting ou^ on ^he roari or Lost out in the road with 
no onf> arounrj for miles 

F Reason can refer to (but not retell) specific places 
m the text for example The letter is not specific 
enough Also referring to the place where specific 
adjectives wore quoted from is merely a circular reason 

4 Character trait identified and substantiated. 
Respondents identify character trait{s) and substantiate 
their chotce(si with at least two reasons or pieces of 
evidence felatecj to thp text However the evidence may 
be presented m an amh-guous fashion or be of the types 
described m 3C 3r Reasons must be distinct — not 
instances of ♦ho same rrason such as It had sad parts 



not any happy parts " This is a restatement of the same 
reason Other instances of single reasons are when it 
takes two bits of information to make a single point, for 
example, (wanted to thinlc things over) "He said he 
wanted to come home, but he didn't thinic he was ready 
for college." 

5 = Character trait identified and substantiated 

In a coherent fashion. Respondents identify character 
trait(c) and substantiate their choice(s) with at least two 
reasons or pieces of evidence clearly related to the text 
— directly related or can be readily inferred The reasons 
are presented logically and coherently 

NOTE. The following types of papers were classified 
as indicated and received no further scoring* 

0 = No response. 

7 = Illegible or Illiterate. 

8 = Totally off task. 

9 = "I don't know." 

M Second Categorization — The source of the 
evidence. Code presence or absence for each of the 
following. 

1 = Content. The evidence is based on the content of 
the text. 

2 = Form. The evidence is based on the language, 
style or construction of the text 

3 = Subjective reactions. These are responses that 
judge the worth of all or part of the text, such as "It was 
interesting" or "it was monotonous " Personal opinions 
about the actions of the characters are stated, such as: 
"David should not have run away" or references to the 
moral of the story or general philosophical statements are 
made 

NOTE Content and form can be present only if 
primary categorization is a "3" through "5", subjective 
reactions can be present m papers categorized "2" 
through "5 " 

III. Third Categorization— A count of the number of 
reasons or pieces of evidence Categorization for the 
count of details is as follows: (1), (2), (3), (4), (5), (6), (7 
or more) NOTE. This count only applies to papers with 
primary categorization of "3" through "5", subjective 
reactions should not be counted as reasons or evidence 

Somebody's Son Exercise: Format 2 
"Somebody's Son" was also presented to 
13- and 17-year-olds with the following 
instructions. 

Read the story below Then write down your thoughts 
and feelings about the story on the lines provided on the 
next two pages We are interested in what you have to 
say. not your spelling and punctuation You will have 9 
minutes to read the story and write your response 
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Responses were scored with the 
following guide. 

Scoring Guide— Responding to 
Literature 

General Responding 
''Somebody'e Son" 
H-402000-B1B-2, 3 
Age 13, Package 9, Exercise 4 

Q«n«ral Scoring Rational*: There are several 
major ways to deepen understanding of a written work 
Respondents can use awareness of emotional irripact, 
personal experience and knowledge of other works to 
interpret, provide meaning, evaluate and analyze the text 
A content analysis of the responses not only provides 
information about which internal resources respondents 
tend to draw upon to help their understanding of written 
works, but also the cognitive skills they choose to 
demonstrate when given an opportunity to respond freely. 
It is expected that the results may be highly text 
dependent Also, the better responses should move 
t>eyond plot summary and retelling to provide meaning, 
evaluation and particularly analysis 

Scoring Quidf Categories: 

Descriptive information Code each type as present or 
absent. Code one type as predominant. 

EQ = Egocentric. Responses are not text based, but 
are text relevant Respondent writes a letter or story o^ his 
own or writes another story (or excerpts) that he hdo 
memorized. Other types of statements categorizes here 
are. "1 never read stories"; Tm not good with stones"; 
or "I'm sorry to run out on you. I don't want tc 
go to college, either." 

PR = Personal. Respondent identifies with 
characters, makes judgments about actuns of characters 
or gives advice, for example. '1 might have done the 
same thing," "David shouldn't have left home." "His 
father should take him back," "Hopetclly his father will 
tie the cloth on the tree." or statements, such as: "1 
like stories like that" or "This not my kind of story." 

EM - Emotional. Respondent attributes emotions 
to the text or makes a direct statement of emotion, for 
example* "The story was sad." "It's touching," "It had 
a funny feeling," "It was very dramatic," or "I felt sorry 
for the boy " 

RT - Retelling. Respondent summarizes or 
paraphrases the story (or parts of it) using specific words 
from the story Respondent gives a synopsis, overview or 
brief description of the story or part(s) of it (Disregard 
inaccuracies ) 

IN ^ Inferenclng. Respondent goes beyond the text 
and provides motivations for characters, for example 
"David learned a lesson/^ "David's parents needed him to 
help pay the bills, ' or David feels that his father doesn't 
love him " 

ON - Generalization. Respondent attributes 
meanings to the story, for example "Go out and try 
new things, " "It shows that people have feelings that 
can be hurt and people are the ones that hurt each 
other," or "Everyone knows you can't run away from 
your problems " 



AN = Analysis. Respondent discusses the language 
and/or structure of the story, for example "It could have 
more details and not so many Jong words. ' "I didn't see 
any misspelled words." "It wasn't long enough." or "The 
author uses imaginative language." 

OW = Other works. Respondent classifies the work 
as to genre or type and compares the story to other works 
or art forms, such is. "It's not like a story I've seen 
before," or " I tfimk it s a good soap opera " 

EV = Evaluation. Respondent judges tho worth of the 
work, for example: "'It was stupid," "I don't like it,'" " I 
didn't understand it." " It doesn't make sense. ' " It is nicely 
written."' "'It was not exciting or sad," 'It has no meaning." 
or 'I! is imaginative." 

MOTE: in addition to the papers which were 
considered ratable (1 = ratable) and which were 
analyzed using the categones described above, some 
papers were not considered ratable, and these were 
placed in one of the following classifications 

0 = No response. 

2 = Nonratable. Copies or circular 

7 ^ Illegible, Illiterate. 

8 = Totally off-task. 

9 = "I don't know." 



2. Good Dog Exercise 

Read the story which begins bek)w and theo answer 
the questions on the four pages folk)wing it 

A Story of a Good Dog 

. A man I used to know very well told me this story. He 
was a very truthful kind of man. but he used to elaborate 
things more than a bit, and perhaps he elaborated a bit 

on this. 

Everybody, said he, has a best cat or dog that remains 
in the mind when other very dear memories have faded, 
and one says with complete assurance. "'That was the 
best dog." There were Tommy and Guzzle and Spot, and 
they were all such good dogs that it is hard to believe 
there could be any better, but my best dog had not got a 
name at all; he answered to a very low whistle. He never 
barked; he never made any noise of any kind, except that, 
now and again, he gave a kind of whispered gargle away 
down in his innards — you could hardly hear it— and 
that showed the top of his delight Poor old fellow, he 
hadn't much to be delighted about he was just delighted 
to be delighted. 

This IS how we met It was evening, and I was 
going up a street and down a street I was looking fur 
something. Among other things I was looking for a job. but 
this time I was looking for something else— *and suddenly 
I saw It It was in a dog s mouth it was nearly half a loaf of 
bread, and the dog was stinking up an alley with it. He 
was a large terrier kind of dog, and I began stalking him 
for my share of whatever he had I stalked him to the end 
of the alley, and he whispered a few very low grunts at 
me 'Drop it, " said I. and he dropped it 
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He sat down a few steps away, and he looked at the 
bread, and he looked at me, and then he scratched 
himself, and then he looked at the bread again I broke 
the half-loaf into fairly even pieces, and I handed him the 
larger half He looked at me, and he looked at the bread, 
and I could see that he was trying to work out where the 
catch was Then he scratched himself with some fury, and 
when i began to eat, he began to eat He would stop 
every now and then to take a good look at me, and then 
he made that curious whispered gargle of delight away 
down in his innards, and then he started again on 
the bread 

How good th'jt bread was' I remember thinking that 
cake was not ^ -ilf as delicious as bread, and that this 
bread ^ z.^ the best that ever was baked. I could have 
wdien a hundred loaves cf it. and then I could have eaten 
the dog 

It was evening and darkish, so I hunched myself up for 
sleep just where I was. and the dog. very cautiously, canne 
to me and at last tightened himself up against me, and he 
gargled a little and scratched himself nearly all night I 
didn t care about anything I had a friend, and he cared 
less for he had a friend too 

In the mornino I saw what al) the scratching was 
about He was covered from head to foot wrth mange He 
was almost a solid mass of scabs One of his eyes was 
blind He was about three years younger than I was — that 
IS he was rather old and he looked at me out of his one 
goodish eye with the kindly adoration that a good dog 
reserves for its best pup He knew that I was an 
incompetent person, and he was very glad of that, for he 
had made up his mmd that he would feed me by day and 
keep me warm by night 

It was AinterfSh and rainish. and darkish, and we 
wandered together up a place and down a place, and we 
kept carelully out of everybody s way I didn t want people 
because they wouldn t give me a jOb. and he didn t want 
people because they heaved rocks at him 

For more than two months, whatever I ate he brought 
Tie He was marvelously skillful He knew where bits of 
bread grew Sometimes there wasn t any but most days 
there was a bit 1 should have been very hungry, but m 
those days my mind moved around at about a mile a 
minute and >t hated stomachs 

One mornmg my good dog set out hunting as usual 
Then having gone but a few steps he stopped He came 
back and pushed hiS head against me Then he moved 
away again and stopped again And then suddenly, he 
lifted his head to the sky and howled —the first sound I 
had ever heard him make' He howled and howled as 
though he we^7 trying to howl himself dead My heart 
near"; burs^ wvith terror I ran lo him and took his head into 
my tirms whispering love words to him and as I looked 
into hiS facp I saw what was wrong 

His other eye was gone He was quite blind He 
rouidn f gr) hunting He wasn t howling about himself he 
was howling because hf couidn t nose out something for 
>e to eat He cared no more about himself than I at that 
moment rurec] at)t)u\ mf He wanted to howl himself 
dead but my arms coaxed him and m a little while he 
stood silPnt and shivenng 



I picked him up — he was not a light dog — and ( 
walked and walked and walked There were fields on one 
side, With a dull sky over them, over us the world was a 
box, and we were two rats in a trap I came to a smalt 
place, and saw a druggist's shop tn it; the door was open 
I put the dog down behind the door and walked in There 
was a man — perhaps ten years older than I — behind the 
counter, and I said to him, "Please, will you kill my dog for 
me, without hurting htm'?" 

He was a hard-faced, tough man, and he looked me 
up and down with eyes that were like bullets 

"What will you pay me with''" said he 
I have no money," I answered 

"I've no painless poison," said he 

I looked at the rows of bottles on the counter and on 
the walls, and my ,iext words were inspired, they were 
even crafty. 

"You are a very wise man," I said There is nothing of 
this kind that you couldn't do ' 

His eyes became eyes again 

"What's wrong with the dog*?" said he 

"He IS old and blind and dying of the mange " 

"Where is the creature'?" 

I pointed and he went to look 

"Good God Almighty." said he. as he stared at the 
Crouching dog "Take that thing away " 

I followed him into the shop 

"Listen. " said I. "that is the best dog in the world He 
has fed me for over two months ' 

"You ate out of that thing's mouth'?" said he 
Yes," I answered "He went blind this morning, and 
he IS dying of horror because he can t feed me " 

The man looked at me as if I were crazy 

' You know how to do everything." I said "Do this for 
the best dog in the world " 

He turned briskly took up a pile of papers, and 
placed them on the Moor Stretch him on that. " he 
Ordered I picked my dog up — he was all one jelly of 
trembling terror 

The man went behind his counter, and in about five 
minutes he came back again, carrying a bowl 

Good God'" said he. as he looked at the dog. and 
then his face went gentle There is warm milk and bread 
arrd sugar — dogs love sugar," he added, "and there is 
just enough of something else that will end his troubles in 
no time ' 

I put the bowl lo my dog's nose He smelled, and 
smelted again, and then^ with an astonishinq jOy, he 
began to lap and eat while I petted him He was tasting a 
kind of food that perhaps he hadn t tasted for six or seven 
years Not since his master of long ago had thrown him 
out, and flung rocks at him till he left 

Take him down to the fields, ' said the man "He'll be 
gone in less than half an hogr ' 

So I sat among the bushes with him, and he began 
doing and undoing things, he began to go aiieep and he 
began to waken up and he began to gargle joyfully, and 
then he began to forget these things, and all things 
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He forgot blindness and age and fear He forgot 
hunger, he forgot me, he forgot to scratch himself, he 
forgot life itself He stretched himself a little, luxuriously, 
and then a small shiver ran all over him, and he 
was gone. 

I pushed htm deeply into the bushes, and walked 
away, pretty lonely again, but I think you will agree that 
when I say he was my best dog, I am not making any 
mistake in that very important matter 

Shortly after that I got a job — said the man who told 
me this story. 

Source Information: "A Story of a Good Dog" as 
reprinted in SEA GREEN HORSE edited by Barbara 
Howes and Gregory Smith was originally published in 
JAMES. SEUMAS AND JACQUES, UNPUBLISHED 
WRITINGS OF JAMES STEPHENS, edited by Lloyd 
Frankenberg Copyright © 1964 by Macmillan Publishing 
Co , Inc. Copyright © 1962. 1964 by Iris Clare Wise. 
Reprinted by permission of Macmillan Publishing Co.. Inc 

A. How was the death of the dog described'^ 

□ As violent and fearful 
D As painful and lonely 

■ As painless and peaceful 

□ As elaborate and ceremonious 
Cj I don't know 

B. In their writing, authors sometimes make use of 
similes — direct comparisons of two unlike things The 
following IS an example of a simile 

The girl ran as fast as a deer 
Which one of the following lines from the story is also 
an example of a simile'^ 

■ . he looked me up and down with eyes that were 
tike bullets 

□ . and thoi'^ I could have eaten the dog. 

□ It was winterish. and rainish, and darkish 

□ . . he was trying to work out where the catch was 

□ I don't know 

C. What kind of person was the man who lived with 
the dog*? 

□ Content and satisfied 

□ Lazy and shiftless 

□ Unfeeling and selfish 

■ Caring and sensitive 

□ I don't *know 

D. This exercise part was not included in analysis 
because no single correct answer was identifiable 

E. In their writing, authors sometirftes make use of 
hyperbole — language characterized by excessive 
exaggeration The following is an example of hyperbold 

The boy was so tall his head touched the clouds 
Which one of the following lines from the story ts also 
an example of hyperbole'^ 

□ Then he scratched himself with some fury, and 
when I began to eat, he began to eat 

rj then he made that curious whispered gdrgle of 
delight away down in his innards, and then he 
started again on the bread 

I ] H^H good thai bread was' I remember thinking that 
cake wa^- not half as delicious as bread 

■ this bread was the best ihat ever was baked. I - 
could have eaten a hundred loaves of it 

r 1 I don't know 



F. What emotions and feelings did you have when 
you read this story*^ Dcscnbe your feelings in a few words 
on the line below. 

G. What was it about the story that made you feel the 
way you did? Write your answer on the lines below 



Scoring Guide — Responding to 
Literature 

Explaining Responses to Literature 

Emotional Responses 
"Good Dog" 
H-403000-B1B-2, 3 
Age 13, Package 13, Exercise 6 
Age 17. Package 13, Exercise 10 

General Scoring Rationale; The responding to 
literature objective was formulated to address deliberate 
conscious kinds of interpretation " It was hoped that 
respondents would not only be aware of their feelings, but 
be confident about expressing them Thus, a successful 
response would both identify an emotion and articulate 
the characters, events and ideas from the text that 
contributed to that emotion. Evidence can also be 
provided from p^r»onal experience, other works or special 
aspects of the construction of the text 

NOTE Rating should, in general, be done by taking 
the entire student response into consideration without 
regard to what is actually wntten in the space orovided for 
the first part and for the second part of the response If 
reasons (substantial) are given m the first part they are 
valid The same is true for emotions or feelings identified 
in the second part Caution should, however be used as 
this often leads to a tendency to rewrite the responses 
Categorizations should reflect as closely as possible 
what respondents actually wrote 

Scoring Guide Categories: 

I. Firat Categorization — Identification and 
substantiation of personal emotions and feelings This 
takes into account both open-ended parts 

1 ^ Unable to identify emotion or feeling. 
Respondents do not do the task They refer to the text, 
but do not answer even the first question Examples are 

A. Some material is quoted from the text with no 
clear identification of the respondent s feeling or emotion 

B. An emotion expressed by a character is identified 
rather than the respondents own feeling for example 

The man was sad 

C. An observation about the story is made such as 
The title is misleading 

2 Emotion or feeling Identified without 
substantiation. Respondents name something but 
cannot go on They identify emotions and feelings but 
do not substantiate choice with evidence from the text 
Respondents tend to provide 1) circular evidence such 
as It s sad because it s sad 2) a copy or close 
paraphrase of the text. 3} vague reasons like The way 
It sounds. The way the author wrote it The words 
used. The way it is made or 4} only a subjective 
reaction as substantiation 
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3 - Emotion or fooNnq Idontlflod and aubatantlatod 
with mlfilmol tvldonco. Roapondonts identify an 
tmotton(8) or fooUng(8) and substantiate their choice(s) 
with only one reason or piece of evidence related to 
the text. 

A. Reason can be directly related to the text, for 
example: "It was sad because the dog died. " 

B. Reason can t>e inferred from the text, such as 
"Sad because the dog was his only friend." 

C. Reason can be inaccurate, if it is ralated^to 
the text. / 

D. Reason can be based on personal experiences or 
opinions that ai e related to the text. / 

E. Reason can refer to (but not retell) specific places 
m the text, such as* "Because of what it says in the 

last paragraph." / 

NOTE "Sorry for the dog" is identificaltion of 
feeling/emotion. "Sorry the dog died" isy^dentification and 
minimal substantiation. / 

4 = Emotion or feeling identified and 
aubatantiated. Respondents identify/ emotion(s) or 
feeling(s) and substantiate their cho(ce(s) with at leaat 
two reasons or pieces of evidenc^Velated to the text 
However, the evidence may be presented in an ambiguous 
fashion or be of the types described in 3C-3E. Reasons 
must be distinct —not mstances^f the same reasons as in 

(Sad) because the dog was starving and hungry " 

5 ^ Emotion or feeling identified and aubatantiated 
In a coherent faahion. Respondents identify e motion (s) 
or feeling(s) and substantiate their choice(s) with at leaat 
two reasons or pieces of evidence clearly related to the 
text — directly related or can be readily inferred The 
reasons are presented logically and coherently 

NOTE The following types of papers were classified 
as indicated and received no further scoring 

0 = No reaponae. 

7 = Illegible or illiterate. 
• = Totally off-taak. 
9 - 'i don't know." 

II. Second Categorization — The source of 
the evidence Code presence or absence for dach of 
the following 

1 Content, i he evidence is based on the content of 
the text, such as The way the dog was moping " 

2 Form. The evidence is based on the language, 
style or construction of the text 

3 . Subjective reactiona. These are responses that 
judge the worth of all or part of the text, for example I 
don t like dog storres I hate to see a dog dead," It 
was interesting or It was monotonous ' Personal 
opinions about the actions of the characters or personal 
expenences nay be stated, such as I had a dog that 
died or references to the moral of the story or general 
philosophical statements may be made, such as Accept 
the death of your dog or Death is a mystery 

NOTE Content and form can be present only if 
primary categorization is a 3 through 5 . subjective 
reactions can be present m papers categorized 2 
through 5 



Third Categorization— A count of the number 
of reasons or pieces of evidence. Categonzation for the 
count of details is as follows: (1)^ (2), (3), (4), (5), (6). (7 
or more). NOTE: This count only applies to papers with 
primary categorization of "3" through "5", subjective 
reactions should not be counted as reasons or evidence. 



3. One of These Days Exercise 

R^d the story which begins below and then answer 
the questions on the three pages following it 

One of These Days 

Monday dawned warm and rainless. Aurelio Escovar, a 
dentist without a degree, and a very early riser, opened 
his office at six. He took some false teeth, stilt mounted in 
their plaster mold, out of the glass case and put on the 
table a fistful of instruments which he arranged in size 
order, as if they were on display. He wore a collarless 
striped shirt, closed at the heck with a golden stud, and 
pants held up by suspenders. He was erect and skinny, 
with a look that rarely corresponded to the situation, the 
way deaf people have of looking. 

When he had thir nged on the table, he pulled 
the drill toward the dental chair and sat down to polish the 
false teeth. He seemed not to be thinking about what he 
was doing, but worked steadily, pumping the drill with his 
feet, even when he didn't need it. 

After eight he stopped for a while to look at the sky 
through the window, and he saw two pensive buzzards 
who were drying themselves in the sun on the ridgepole of 
the house next door. He went on working with the idea 
that before lunch it would ram again. The shrilt voice of his 
eleven- year-old son interrupted his concentration. 

"Papa " 

"Whaf?" 

"The Mayor wants to know if you'll pull his tooth." 
"Tell him I'm not here. " 

He was polishing a gold tooth. He held it at arm's 
length, and examined it with his eyes half closed. His son 
shouted again from the tittle waiting room 

"He says you are, too. because he can hear you, ' 

The dentist kept examining the tooth. Only when he 
had put It on the table with the finished work did he say. 

'So much the better " 

He operated the drill again. He took several pieces of 
a bridge out of a cardboard box where he kept the things 
he still had to do and began to polish the gold. 

Papa ' 

Whaf? 

He still hadn't changed his expression 
He says if you don't take out his tooth, he'll shoot 

you 

Without hurrying, with an extremely tranquil 
movement, he stopped pedaling the drill, pushed it away 
from the chair, and puited the lower drawer of the table all 
the way out There was a revolver ' 0 K . " he said 'Tell 
him to come and shoot me " 
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He rolled the chair over opposite the door, his hand 
resting on the edge of the drawer The Mayor appeared at 
the door. He had shaved the left side of his face, but the 
other side, swollen and m pain, had a five-day-old beard 
The dentist saw many nights of desperation in his dull 
•yes. He closed the drawer with his fingertips and said 
softly; 

"Sit down " 

"Good morning,^' said the Mayor 
"Morning," said the dentist 

While the instruments were boiling, the Mayor leaned 
hiv si''jll or; the headrest of the chair and felt better His 
breath ^'.ds icy. It was a poor office: an old wooden chair, 
..le pedal drill, a glass case w<th ceramic bottles Opposite 
the chair was a window with a shoulder-high cloth curiam 
When he felt the dentist approach, the Mayor braced his 
heels and opened his mouth 

Aureiio Escovar turned his head toward the light After 
inspecting the infected tooth, he closed the Mayor's jaw 
with a cautious pressure >f his fingers 

'"It has to be without an; sthesia." he said 

"Why?' 

'"Because you have an . Vjess " 

The Mayor looked him m the eye "All right. " he said, 
and tned to smile. The dbntlst 'ltd not return the smile 
He brought the basm of sterilized instruments to the 
worktable and took t:iem out of the water with a pair of 
cold twee^^rs. «ti;, without hurrying Then he pushed the 
spittoon with the tip uf his shoe, and went to wash his 
hands in the washbasin. He did all this without looking at 
the Mayor But the Mayor didn't take his eyes off him 

It was a lower wisdom tooth The dentist spread his 
feet and grasped the tooth with the hot forceps The 
Mayor seized the arms of tho chair, braced his feet with all 
his strength, and felt an icy void in his kidneys, but didn't 
make a sound. The dentist moved only his wrist Without 
rancor, rather with a bitter tenderness, he said 

•Now you'll pay for our twenty dead men ' 

The Mayor felt the crunch of bones m his jaw, and his 
eyes filied with tears But he didn't breathe unUI he felt 
the tooth come out. Then he saw it tnrough his tears It 
seemed so foreign to his pam that he failed to understand 
his torture of the five previous nights 

Bent over the spittoon, sweating, panting, he 
unbuttoned his tunic and reached for the handkerchief in 
his pants pocket The dentist gave him a clean cloth 

"Dry your tears. " he said 

The Mayor did He was trembling Whtte the dpntist 
washed his hands, he saw the crumbling' ceiling and a 
dusty spider web with spider's eggs and dead insects 
The dentist returned, drying his hands "Go to bed, ' he 
said, "and ga.g'e with salt water " The Mayor stood up. 
said goodbye with a casual military salute, and walked 
toward the door, stretching his legs, without buttoning up 
his tunic 

"Send the bill. " he said 

"To you or the town"? 

The Mayor didn't look at him He closed the door and 
said through the screen 
"I* the *"ame thing ' 
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Source Information: "One of These Days" from NO 
ONE WRITES TO THE COLONEL and Other Stones, by 
Gabnel Garcia Marquez. translated from the Spanish by 
J.S Bernstein. Copyright © 1968 in the English 
translation by Harper & Row, Publishers, Inc. Repmted 
by permission of the publisher. 

A. How does the dentist feel toward the Mayor'' 

□ Respectful and courteous 

n Concerned and understanding 

□ Fearful and oowerless 

■ Resentful and bitter 

□ I don't know 

8. In the story the son and the dentist talk to 
one anothe 
"Papa 
"Whaf?' 

"The Mayor wants to know if you ll pull his tooth " 
"Tell him I'm not here " 

"He says you are. too, because he can hear you. " 
"So much the better " 

Why did tha ueiiiiCt give this last response'' 

□ Because he misunderstood his son's comment 

□ Because h^ wanted the Mayor to stop complaining 
abou' the pain 

□ Because he was embarrassed that the Mayor heard 
him 

■ Because he wanted the Mayor to know he was lying 

□ I don't know. 

C. Why did the Mayor keep his eyes on the dentist 
while the dentist was gett'ng ready to pull the Mayor's 
tooth? 

□ Because tht Mayor was impressed by the dentist 

■ Because the Mayor did not trust the dentist 

G Because the Mayor was following the instructions 
of the denlist 

□ Because titc Mayor was wait'ng for the anesthetic 

□ I don't know 

a What was the dentist's office like'' 
J Modern and sterile 

■ Poor and untidy 

? Unfurnished and dull 
L~ Cheerful and comfortable 
L I don't know 

E. What did the dentist mention while he was pulling 
the Mayor s tooth'' 

The bill for the treatment 
r The revolver hidden in the table drawer 

■ The twenty deao men 

The buzzards outside the window 
I don t know 

F. Is this a good story'' 
Yes 

No 

G. What was tt about the story that led you to choose 
the answer you did in Question F'' Write your answer on 
the lines below 
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Seeding Guide — Responding 
to Literature 
Evaluating Literature 

Applying Criteria to Evaluate Stories 

"One of These Days'* 

H-420000-B1B-3 

Age 17, Package 7. Exercise 5 

General Scoring Rationale: Since the objective 
referring to the evaluation of written works states that 
It IS important that readers be able to articulate their 
criteria, respondents should explain the rea.'^ns or 
Criteria for their evaluation To be successful, responses 
should provide examples from the text that relate to 
those criteria Plot summary can be viewed as minimal 
support for various criteria, however, the best papers 
should also consider such aspects ^f the text as setting, 
piot and character development, meaning/message, 
clarity of language, relevance or believabitity 

Scoring Guide Categories: 

I. First Categorization — Presentation and 
elaboration of evidence 

1 No criteria or evidence given. Respondent 
copies part of the text or gives a close paraphrase or 
circular response, such as It was good because it was 
good I liked it. I didn t like it," or "I've heard it 
before Nonsensical, or wildly inaccurate statements 
are given 

2 Gives a vague or unelaborated criterion. A 

broad sweeping generalization or personal assbrtion is 
made which does not necessanty have to restate t'le 
phrase It was good bad-- " This response almost ooutd 
have been given m absence of having heard or read ihe 
story It could apply to almost any story It was— excitinq. 
interesting had a good plot, and so on (broad 
general ad}ect>vesi 

3 Retells or gives summary or one vague 
criterion with synopsis as evidence. The summary may 
refer to pa't or all of the story, it may be cryptic or lengthy 
and well written This includes any citing of content of 
story (as long as it is rrot basically copying ) 

' 4 Gives two or more unelaborated criteria. 
Responses contain twD or more generalizations or 
personal assertions (These are longer 2s ") 

5 Gives one criterion elaborated with evidence. 
Respondent gjves one criterion, generalization or 
personal assertion ^^^at is supported with evidence other 
than retelling or plot summary It may or may not be 
accompanied by unelaborated criteria (It was interesting 
because respondent gives something other than 
plot summary ) 



6 Gives two criteria elaborated with evidence. 

Respondent gives two or more criteria, generalizations or 
personal assertions at least two of which are supported 
with evidence other than retelling or plot summary. These 
may or may not be accompanied by unelaborated criteria. 
NOTE. Once a paper meets the criteria listed for a "4." 
"5" or "6" it does not matter if that response is also 
accompanied by plot summary 

NOTE. The following types of papers were classified 
as indicated and receiver' no further scoring 

0 = No response. 

7 = Illegible or Illiterate, 
a = Totally off-task. 

9 - "I don't know." 

II. Second 9'tegorization — Basis of evidence 
Code presence or absence for each of the following. 

1 Content. The evidence is based on the content of 
the text, for example "Gives an idea of the old man's way 
of life ' 

2 - Form. The evidence is based on the language, 
style or construction of the text, for example. It didn't 
seem to h^ve a beginning or an end " 

3 ~ Subjective reactions. These are responses that 
judge the worth of ail or part of the text, or personal 
opinions about the actions of the characters, the 
believability of the pict, the moral of the story, or the 
genre, such as "I Uke fairy tales " 

NOTE" Second categorization is only for papers with 
primary categorizations of ' 2" through 6." 

4. Old Dog Exercise 

Read the poem below and then answer the questions 
on the next two pages 

Old Dog 

Toward the last in the morning she could not get up, 
even when I rattled her pan. 

I helped her into the yard, but she stumbled and fell I 
kne\N It was time. 

The last night a mist drifted over the fields 
In the morning she would not raise her head — the far, 
clear mountains we had wailced surged baci( to mind 
We looiced a slow bargain our days together were the 
ones we had already had 

I gave her something the vet had given, and patted her 
still, a good last friend 

Source Information: "Old Dog" p 205 In STORIES 
THAT COULD BE TRUE by William Stafford Copyright 
< 1971 by William Stafford Reprinted by permission of 
Harper & Row, Publishers. Inc 

A. How much did you liice reh*>tiig this poem'? 

r ' I itlced It very much 

. ' I liiced It 
2 I can't decide 
^ I I di£lti(ed It 

{ I dislii(ed It very much 

NOTE This version with 5 foil options was 
administered only to age 13. at cge 9. only the 3 middle 
foil values were administered For analysis purposes, the 
5 options were combined and renumbered as indicated 
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B. How does the person in the poem feel about the 
death of the old dog'' 

■ The person accepts the dog's death 

□ The person feels angry 

□ The person feels afraid 

n The person has no feeling about the dog's death 
CJ I don't know 

C. How is death described rn the poem'' 
fJ As violent and tragic 

Ci As sudden and shocking 

■ As natural a\ peaceful 

[ i As unfamiliar c '^d strange 
[J I don't know 

D. What emotions arrd .selings did you have when 
you read this poem'' Describe your feelings in a few 
words on the line below 

E. What was it about the poem mat made you feel 
the way you did'' Write your answer on the lines below 



Scoring Guide — Responding to 
Literature 

Explaining Responses 
to Literature 

Emotional Responses 
"Old Dog" 
H-465000-B1B— 1, 2 
Age 9, Package 5, Exercise 7 
Age 13, Package 5, Exercise 4 

Q«n«r«t Scoring Rationale; The responding ta 
literature objective was formulated to adtlress deliberate, 
conscious kinds of interpretation " It was hoped that 
respondents would not only be aware of their feelings, but 
be confident about expressing them Thus, a successful 
response would both identify an emotion and articulate 
the characters, events and ideas from the text that 
contributed to that emotion Evidence can also be 
provided from personal experience, other works or specidl 
aspects of the construction of the text 

NOTE Rating should, in genfc'al. be done by taking 
the entire student response into consideration without 
regard to what is actually written in the space provided for 
the first part and for the second part of the response If 
reasons (substantial) are given m the first part they are 
valid The same is true for emotions or feelings identified 
in the second part Caution should, however, be used, as 
this often leads to a tendency to rewrite the responses 
Categorizations should reflect, as closely as possible, 
what respondents actually wrote 

Scoring Quida Catagorlat: 

t. FIrat Catagorizatlon — Identification and 
Substantiation of personal emotions and feelings This 
takes into account both open-ended parts 

1 - Unabta to idanttfy amotion or faaling. 
Respondents do not do the task They refer to the text 
but do not answer even the first question Examples are 



A. Some material is quoted from the text with no 
clear identification of the respondent's feeling or emotion 

B. An emotion expressed by a character is identified 
rather than the respondent s own feeling, for example 
"The man was sad " 

C. An observation about the story is made The 
title is misleading " 

2 ^ Emotion or feating idantifiad without 
aubatantlatlcn* Respondents name something but 
cannot go on They identify emotion(s) and feeling(s) but 
do not substantiate choice(s) with evidence from the text 
Responses tend to provide 1) circular evidence such as 
"It's sad because it's sad", 2) a copy or close paraphrase 
of the text, 3) vague reasons like The way it sounds. 

' The way the author wrote it, ' "The words used. The 
way It IS made", or 4) only a subjective reaction as 
substantiation 

3 = Emotion or faeting Idantifiad and substantiated 
with minimal avidanca. Respondents identify emotionis) 
or feeling(s) and substantiate their choice(s) with only 
one reason or piece of evidence related to the text 

A. Reason can be directly related to the text, fc 
example It was sad because the dog died ' 

B. Reason can be inferred from the text, for example 
'Sad because the dog was her only friend 

C. Reason can be inaccurate, if it is related to 
the text 

D. Reason can be based on personal experiences or 
opinions that are related to the text 

E. Reason can refer to (but not retail) specific places 
in the text, such as Because of what it says in the 

last paragraph ' 

NOTE Sorry for the dog is identification of 
feeling/emotion Sorry the dog died is identification and 
minimal substantiation 

4 ^ Emotion or failing identified and 
aubatantlatad. Respondents identify emotion(s) or 
feeiing(s) and substantiate their choice(s) with at laaat 
two reasons or pieces of evidence related to the text 
However, the evider .e may be presented in an 
ambiguous fashion or be of the types described m 3C-3E 
Reasons must be distinct — not instances of the same 
reasons as in (Sad) because the dOg was aged and old 

5 Emotion or feating idantifiad and substantiated 
in a coharant fashion. Respondents identify emotion(s) 
or feeling(s) and substantiate their choice(s) with at least 
two reasons or pieces of evidence clearly related to 

the text — directly related or can be readily inferred The 
reasons are presented logically and coherently 

NOTE The following types of papers were classified 

indicated and received no further scoring 

0 No response. 

7 Illegible or iiliterata 

6 Totally off-task. 
9 'I don't know. ' 

H. Second Categorization — The source of the 
evidence Code presence or absence of each of 
the following 



1 - Conttnt. The evidence is based on the content of 
the text, for example* "About the mist over the field." 

2 - Form. The evidence is based on the language, 
style or construction of the text 

3 ^ Subfective reactions. These are responses 
/hat judge the worth of all or part of the text, such as: 

I don't like dog stones, " "I hate lo see a dog dead," "It 
was interesting." or ' It was monotonous " Personal 
opinions about the actions of the characters or personal 
experiences may be stated, such as "I had a dog that 
died" or references to the moral of the story or general 
philosophical statements are made, such as. "Accept the 
death of your dog." or Death is a mystery" 

III. Third Categorization — A count of the number 
of reasons or pieces of evidence Categonzation for the 
count of details is as follows (1), (2), (3), (4), (5), (6), (7 
or more) NOTE This count only applies to papers with 
primary categorization of 3" through "5", subjective 
reactions should not be counted as reasons or evidence 

5. Mother to Son Exercise 

Read »he poem below and then answer the questions 
on the next two pages 



Mother to Son 

Well son. I II tell you 

Life for me am t been no crystal stair 

It s had tacks m it. 

And splinters, 

And boards torn up. 

And places with no carpet on the floor- 
Bare 

But all the time 

j I se been a-ciimbm on 

i And reachin landm s 

i And turn'n corners 
And sometimes goin in the dark 
Where there am t been no light 
So boy don t you turn back 
Don t you set down on the steps 
Cause you fmds its kmder hard 

! Don r you fall now — 

I For I se still gom honey 

I I s still ciimbm 

I And life for mti am t been no crystal stair 

L . 



Source Information Mother to Son by Langston 

Hughes Copyright > i926 by Alfred A Knopf Inc and 
rf»newf^tl ^0S4 by Lanqston Hughe^ Reprmted from 
SMFCrfcO POEMS OF LANGSTON HUGHES, by 

p»vnissir^n ■ .f Aifr^'d A Knopf !nc 

A H -w doT's thf rnother ^alk to her son'' 

■ [ fu DuraqiMqIy 
Jf>King!y 
Hopt'lpssiy 



B. This exercise part was not included m analysis 
because no single correct response was identifiable 

C. Is this a good poem? 

□ Yes ■ 

□ No 

□ No 

D. What was it about the poem that led you to 
choose the answer you did in Question Write your 
answer on the lines below 



Scoring Guide — Responding 
to Literature 
Evaluating Literature 

Applying Criteria to Evaluate Poems 
"IMother to Son" 
H-467000-B1B-2, 3 
Age 13, Package 9, Exercise 7 
Age 17, Package 9, Exercise 8 

General Scoring Rationale: Since the objective 
refernng to the evaluation of written works statec *hat 
"It IS important that readers be able to articulate their 
criteria. " respondents should explain the reasons or 
criteria for their evaluation To be successful, responses 
should provide examples from the text that relate to those 
criteria. Plot summary can be viewed as minimal support 
for various criteria, however, the best papers should also 
consider iuch aspects of the text as setting, plot and 
character development, meaning/message, clanty of 
language, relevance or believability 

Scoring Guide Categories: 

I. First Categorization — Presentation and 
elaboration of evidence 

1 = No criteria or evidence given. Respondent 
copies part of the iext or gives a close paraphrase or 
circular response, for example: "It was good because it 
was good," "I liked it," "1 didn't like it." or " I've heard it 
before " Nonsensical, or wildly inaccurate statements 
are given 

2 = Gives a vague or uneiaborated criterion. A 

broad, sweeping generalization or personal assertion is 
made which does not necessarily have to restate the 
phrase 'It was good/bad— " This response almc^t could 
have been given in absence of having heard or read the 
poem It could apply to almost any poem It was — 
exciting, interesting, had a good plot, and so on (broad 
general adjectives) 

3 Retells or gives summary or one vague 
criterion with synopsis as evidence. The summary may 
refer to part or all of the poem, it may be crypt«c or lengthy 
and well written This includes any citing of content of 
poem (as long as it is not basically copying) 

4 Gives two or more uneiaborated criteria. 
Responses contain two or more generalizations or 
personal .insertions (These are longer 2s ) 
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5 » QlvM OM criterion •laborattd with •vlil«nc«. 

Respondent gives one citerion. generalization or 
personal assertion ttiat is supported with evidence other 
than retellirtg or plot summary; It may or may not be 
accompanied by uneiaborated criteria. (It was interesting 
because . • . ; respondent gives something other than 
plot summary.) 

6 » Qlvee two criteria tlaborated wHh evidence. 
Respondent gives two or more cnteria, generalizations or 
personal assertions at leaat two of which are supported 
with evidence other than retelling or plot summary; these 
may or may not be accompanied by uneiaborated cnteria. 
NOTE: Once a paper meets the cnteria listed for a "4, " 
"5" or "6" it does not matter if that response is also 
accompanied by plot summary. 

NOTE: The following types of papers were classified 
as indicated and received no further scoring: 

0 ^ Mo reeponee. 

7 = Illegible or Illiterate. 

8 = Totally off-taak. 

9 = don't know/' 

II. Second Categorization — Basis of evidence. 
Code presence or absence for each of the following: 

1 = Content. The evidence is based on the content of 
the text, for example: "It wa«^ about the crystal stair." 

2 = Form. The evidence is based on the language, 
style or construction of the text, for example: "There was 
so many misspelled words" or 'Poems are supposed 

to rhyme." • 

3 = Subjective reactlona. These are responses that 
judge the worth of all or part of the text, or give personal 
opinions about the actions of the characters, the 
believability of the plot, the moral of the poem, or the 
genre, such as: "I like poetry. ' 

NOTE: Second categorization is only for papers with 
primary categorizations of "2" through "6 " 

6. i Was You Exercise 

Read the poem below. Then write an essay about 
an important idea or theme of the poem. In your essay 
tell how such things as the i'^'ages. events, sound 
and structure conmbute to this idea or theme. We are 
interested in what you have to say not your spelling and 
pu.ictuation. Write your essay on the lines provided on the 
next two pages. You will have 9 minutes to read the poem 
and writs your response 



1 was you 

I smiled 
your smile ' 
till my mouth 
was set 
and my face 
was tight 
and It wasn't nght 
It was wrong 
I was you baby 
I was you too long 

^ said 
your words 
till my throat 
closed up 
and i had 
no voice 
and I had 
no choice 

but to do your song 
I was you baby 
I was you too long 

I lived o 
your life 
till there was 
no me 
I was flesh 
I was hair 
but I wasn't there 
It was wrong 
I was you baby 
I v^as you too long 
and baby baby 
the worst thing 
to It 

IS that you let me 
do It 

so who was weak 
and who was strong 
for too long baby 

Source Information: ^ i was you from ON MY WAY 
TO WHERE by Don Previn published by Saturday Review 
Press. 1972 Used by permission of the publisher 



Scoring Guide — Responding to 
Literature 

Analyzing Literature 

"i was you*' 

H-469000*BlB-3 

Age 17, Package 5, Exercise 8 

General Scoring Rationale: The objectives state 
that students should be able to turn to a text and attend 
to special aspects such as the stylistic conventions 
employed the format, structure or areas of multiple 
meaning A successful analysis goes beyond 
interpretation, providing ^ theme or meaning, and 
discusses in what way particular features — images, 
sounds events and stn^cture --of the poem contnbute 
to the theme 



Scoring Quid« Cat«gorias: 

\ = No analysis. These responses only evaluate 
the poem or its features or make empty or glancing 
references to various features Examples are * All and all 
this poem was pretty and I enjoyed it," "It did have a lot of 
phrases that rhymed, " The poem presented poor images 
and events. " 'The structure was catchy " or ' The sound 
IS your singing a song " 

Some category "t" responses do include brief 
allusions to the poem However, these mentions of text 
are not considered synopsis Also, wildly inaccurate 
interpretations of the poem and nonsensical responses 
should be placed in this category 

2 - Synopsis. These responses mainly retell or 
summarize the poem Although some may include 
evaluations and empty or glancing references to other 
features, sometimes a brief synopsis can be embedded in 
an evatuatton If so. place it in category '2 " The same is 
true of some references to images When the meaning of 
an image is not given, but part of the poem is repeated, 
then the response can be placed in category "2 " Also, 
responses that include glancing references to a number 
of features including events should be placed in this 
category In summary, a '2" response at least retells, 
summarizes, or refers to particular parts of the poem 
However it does little else of substance in terms cf 
analyzing the poem 

3 Theme, inese responses state an idea or theme 
of the poem but do fittle of substance They do not 
include synopsis or relevant discussions of other features 
Some may include evaluations, glancing references to 
features or philosophizing about their theme that is not 
parttcularly relevant to the poem In other words, some 

3 responses may go off on tangents (initiated by the 
theme) Which are not text based A paper with no theme 
statement but a substantive statement of one feature 
other than events should also be placed in this category 
For example An image is given Some themes are It 
presented the idea of weakness in people The basic 
Idea of the poem is how love hurts when misused or 
m.shandied The theme is that you should not try to 
be somebody else or Always be your own person 
With a poem in particular the difference between 

synopsis and theme is often a fine distinction Yet 
the basic difference is whether or not the idea message 
IS stated as a generalization 5,ynopsis can involve 
hypotheses about meaning yei th»s is usually 
interpretation not generalization, for example I thmk it 
means he she lost his herjdentrty Also some 3 
responses elaborate their generalization to the point of 
directly referring to parts of the poem such as By living 
by someone else s feelings vews likes and dislikes 
talKS and walks you do not hav*^ an identity of your own 
However these should still be placed in category 3 

4 Minimal evidence of analysis. Some of these 
responses state an idea or theme of the poem and relate 
events in the poem (piot'summaries nnay t)e quite thin) 
References to specific parts of the text qualify as 
synopsis Other responses placed m this category discuss 
at least two features one ran t)e events (synopsis) in 
a substantive manner however there is no statement 
of theme 
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5 = Evidence of anaiysis. These responses state 
an idea or theme of the poem and include a substantive 
statement about at isast one feature other than events 
(synopsis) For example, the structure might be 
discussed, such as 'The short lines and the choice of 
words suggest that this person is still not himself" or 'The 
striiCture of this poem brought a melody of music, as 
though It were the lyrics of a soft- spoken song of tearful 
anger " These responses may also incluue synopsis or any 
of the other characteristics of papers placed in categones 
"1-4 ' However, the major drawback with category "5" 
papers is that they may be brief (include only one 
substantive statement), or, if they do include discussions 
of several features, the features explained or even 
elaborated do not all relate or refer back to the idea/theme 
proposed in the response The relationship between the 
stated theme or :dea of the poem and the discussion of 
the features is not explicit or even readily implicit 

6 ^ integrated anaiysis. These responses state an 
idea/theme and discuss at ieast two features, one can be 
events (synopsis), in a substantive manner They often 
have the components of ' 5" papers, yet the discussion of 
the features does relate to the proposed idea/theme 
These are coherent, organized responses 

NOTE The following types of papers were classified 
as indicated and received no further scoring 
0 ^ No response. 

7 - ilieglbie or illiterate. 

8 - Totally off-task. 

9 - "i don't know.*' 

7. What Makes a Good Story 
Exercise 

What makes a good story*? List three things on the 
lines below 



2 
3 



Sparing Guide — Responding 
to Literature 
Evaluating Literature 

Qualities of Good Literature 
'Good Story" 
H-841000.B1B-1, 2,3 
Age 9, Package 11, Exercise 4 
Age 13, Package 14, Exercise 5 
Age 17, Package 10» Exercise 5 

Scoring Guide Cstegories: 
I First Categorization ~ Description of type of 
qualities listed 

0 No response. 

1 Relationship between form and content 

Respondents may state that the dialogue is compatible 
wjth the lopir for example 
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2 = Content. Respondents may refer to one or more 
of the following types of content mystery, westerns, 
fantasy, adventure, danger, action, humor, suspense, 
romance, drama any reference to theme, or excitement 

3 - Form. Respondents mention some aspect of 
form, such as high point, strong words, vivid language, 
suspensefut beginning, length, style, construction of the 
text, or happy ending. 

4 = Subjective reaction. Respondents give a 
statement to the effect that a good story should evoke a 
subjective reaction to one of the following types' sensible, 
interesting, intelligent, funny, imaginative, dramatic, 
suspenseful, or adventurous 

5 - Unelaborated features of genre. Respondents 
refer to one of the following characteristics: plot, 
character, setting — with or without redundant "good " 

6 ^ Nemlim of a specific story or author 
Respondents list a particular title or author, 

7 ^ Undetermined or circular. These are responses 
where you cannot determine whether the quality -is one 
of content or form, or the answer is circular, for example 
the writer, author, good author, good literature, good 
writing, good words, the title, the ending, language, or 
good subject 

8 ^ References to format. Respondents list some 
quality related to format, for example neatness, commas, 
quotation, indentations, capital letters, summary, 
controlling idea, or has a title 

9 = Other. Responses are totally off task, tllegible. 
illiterate, I don t know. ' or other nonsense 

II. Second Categorization — Level of the qualities 
.ated 

1 ^ Identifies characteristics of worl( as a whole. 

Responses refer to sex, violence, human adventure, 
catchy title, dialogue, plot, or setting 

2 = Analyzes the way the text works. Respondent 
states ideas, such as use of foreshadowing or irony, 
sentences are to the point, the tension rises, enthusiastic 
words are used, surprise ending is effective 

3 MaKes statements about the meaning or 
theme. Respondent makes statements, such as "It 
makes me think' , "It has a good moral' , It expresses 
your feelings ", or It has meaning, meaningfu iess " 

NOTE This categorization only applies to papers 
rated 1" through 5" for the first categorization 

8. What Makes a Good Poem 
Exercise 

What makes a good poem'' List three things on the 
lines below 

1 ^ 

2 



Scoring Guide— Responding 
to Literature 
Evaiuating Literature 

Qualities of Good Literature 
"Good Poem" 
H-842000-B1B-1,2, 3 
Age 9, Package 10, Exercise 8 
Age 13, Package 8, Exercise 10 
Age 17, Package 8, Exercise 8 

Scoring (luide Categories: 

I, First Categorization — Description of type of 
qualities listed 

0 = No rvsponae. 

1 - Relationship between form and content. 

Respondents may state the rhyme pattern is compatible 
with the topic, for example 

2 ^ Content Respondents may refer to one or more 
of the following types of content mystery, westerns, 
fantasy, adventure, danger, action, humor, suspense, 
romance, drama, any reference to theme, or excitement 

3 - Form, Respondents mention some aspect of form, 
such as* good rhyme, high point, strong words, vivid 
language, suspenseful beginning, length, style, 
construction of the text, or happy ending. 

4 = Sub|ectlve reaction. Respondents give a 
statement to the effect that a poem should evoke a 
subjective reaction of one of the following types sensible 
interesting, intelligent, funny, imaginative, dramatic, 
suspenseful. or adventurous. 

5 = Unelaborated features of genre. Respondents 
refer to one of the following characteristics plot, 
character, setting — with or without redundant good 

6 ^ Naming of a specific poem or poet. 
Respondents list a particular poem or poet 

7 = Undetermined or circular. These are responses 
where you cannot determine whether the quality is one of 
content or form, or the answer is circular, for example 
poet, good poet, good literature, good writing good 
words, the title, the ending, language or good subject 

ft ^ References to format. Respondents hst some 
quality related to format, for example neatness commas 
quotation, indentations, capital letters summary 
controlling idea, or has a title 

9 Other Responses are totally off task illegible 
Illiterate. I don t know or otner nonsense 

II. Second Categorization --Level of the qualities 
listed 

1 ^ Identifies characteristics of work as a whole. 
Responses refer to sex violence human adventure 
catchy title, dialogue good rhythm plot or setting 

2 ' Analyzes the way the text worKs. Respondent 
states ideas, such as use of fordshadowtng ur irony the 
tension rises, enthusiastic words are used or surprise 
ending is effective 

3 MaKes statements about the meaning or 
theme. Respondent makes statements such as It 
makes me think , It has a good moral It expresses 
your feelings or It has meaning meaningfuiness 

NOTE TniS categorization only applies to papers 
rated 1 through 5 for the first categorization 
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9. Check Exercise 

You are going to be asked to write a composition about 
a poem I will read the poem to you as you read it to 
yourself. When we have read the poem carefully, write a 
composition in which you discuss the poem We are more 
interested in what you have to say than in how well you 
say It Put the title of the poem at the top of the next page 

Check 

The Night was creeping on the ground' 
She crept and did not make a sound. 

Until she reached the tree And then 
She covered it. and stole again 

Along the grass beside the walM 
— I heard the rustling of her shawl 

As she threw blackness everywhere 
Along the sky. the ground, the air. 

And in the room where I was hid! 
But, no matter what she did 

To everything that was without 
She could not put my candle out! 

So I stared at the Night And she 
Stared back solemnly at me< 

James Stephens 



Source Information: "Check" by James Stephens 
Used by permission of Macmillan Publishing Co.. Inc.. 
from COLLECTED POEMS of James Stephens Copyright 
© 1915 by Macmillan Publishing Co . Inc renewed 1943 
by James Stephens 



Scoring Guide— Responding 
to Literature 

General Reeponding 

"Check" 
4-200005-226-2 

Age 13, Package 10, Exercise 8 

General Scoring Rationale: There are several 
major ways to deepen understanding of a written work 
Respondents can use awareness of emotional impact, 
personal experience and knowledge of other works to 
interpret, provide meaning, evaluate and analyze the text 
A content analysis of the responses not only provides 
mformation about which internal resources respondents 
tend to draw upon to help their understanding of written 
works, but also the cognitive skills they choose to 
demonstrate when given an opportunity to respond 
freely It is expected that the results may be highly text 
dependent Also, the better responses should move 
beyond plot summary and retelling to provide meaning, 
evaluation and particularly analysis 
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Scoring Guide Categories: 

Descriptive information Code each type as present or 
absent Code one type as predominant 

EG = Egocentric. Responses are not text based, but 
are text relevant Respondent writes a letter or poem of 
his own or writes another poem (or excerpts) that he has 
memorized. Other types of statements categonzed here 
are "I never read poetry." "I'm not good with poems." 
or "A check on a piece of paper because he got the 
answer wrong " 

PR = Personal. Respondent identifies with 
characters, makes judgments about actions of characters 
or gives advice, for example "f feel that the poet is right. ' 
Also, statements are made such as "I like poems about 
nature." "I wish I could write a poem like that." or "This is 
not my kind of poem." 

EM Emotional. Respondent attributes emotions 
to the text or makes a direct statement of emotion, for 
example. "The poem was sad." "It's touching/' "It had a 
funny feeling." "It was very dramatic." "It was a spooky 
poem." or "It was a mysterious poem." 

RT = Retelling. Respondent summarizes or 
paraphrases the poem (or parts of it) using specific words 
from the poem Respondent gives a synopsis, overview of 
brief descnption of the poem or part(s) of it. (Disregard 
inaccuracies.) 

IN ^ Inferenclng. Respondent goes beyond the text 
and provides motivations for charactAf's 

QN ^ Generalization. Respondent attributes 
meanings to the poem, such as. "You shouldn't close your 
mind to anything unknown," or "It means that the world is 
suddenly a check of darkness." 

AN - Analysis. Respondent discusses the language 
or structure of the poem, for example: "The poem doesn't 
rhyme." "The poem seems more like a story." "The night 
seemed like a witch," "It could have more details and not 
so many long words." "The author makes the night sound 
like a real person you could reach out and touch." "I didn't 
see any misspelled words. " or "It wasn't long enough." 

OW = Other works. Respondent classifies the work 
as to genre or type. Respondent compares the poem to 
other works or art forms, such as. "It's not like a poem 
I've seen before. ' "I think it's a good soap opera. ' or "It 
was like a mystery." 

EV = Evaluation. Respondent judges the worth of 
the work, for example. "It was stupid/ "I don't like it." 
I didn't understand it." "It doesn't make sense. " "It is 
nicely written. ' "It was not exuting or sad." or "It 
has no meaning " 

NOTE In addition to the papers which were 
considered ratable (1 ^ ratable) and which were analyzed 
using the categones described above, some papers were 
not considered ratable and these were placed in one of 
the following classifications 

0 = No response. 

2 - Nonratable. Copies or circular 

7 = Illegible. Illiterate. 

8 Totally off-tatk. 
^ '1 don't know. " 




10. Into My Heart Exercise 

Write a composition m which you discuss th|b poem. 
We are more interested m what you have to say than in 
how you say it 



Into my heart an air that kills 

From yon far country blows 

What are those blue remembered hills. 

What spires, what farms are those'' 

That IS the land of lost content, 
i see It shining plain. 
The happy highways where 1 went 
And cannot come aga'n 



A E Housman. 1890 



Sourc« tnTormatlon: Excerpt from "A Shropshire 
Lad *— Authorized Edition— from THE COLLECTED 
POEMS OF A E HOUSMAN Copyright 1939. 1940. 
1966 by Holt. Rinehart and Winston Copyright f) 1967. 
1968 by Robert E. Symons. Reprinted by permission of 
Holt. Rinehart and Winston. Publishers 



Scoring Guide — Responding 
to Literature 

General Responding 
''into My Heart" 
4-200008-22B-3 

Age 17, Package 10, Exercise 10 



General Scoring Rationale: There are several 
mafor ways to deepen understanding of a written work 
Respondents can use awareness of emotional impact, 
personal experience and knowledge of other works to 
interpret, provide meaning, evaluate and analyze the text 
A content analysis of the responses not only provides 
information about which internal resources respondents 
tend to draw upon to help their understanding of written 
works, but also the cognitive skills they choose to 
demonstrate when given an opp ')rtunity to respond 
freely It is expected that the results may be highly text 
dependent. Also, the better responses should move 
beyond plot summary and retelling to provide meaning, 
evaluation and particularly analysis 

Scoring Guide Categories: 

Descriptive information Code each type as present or 
absent Code one type as predominant 

EG - Egocentric. Responses are not text based, but 
are text relevant Respondent writes a letter or poem of 
his own or wntes another poem (or excerpts) that he has 
memorized Other types of statements categorized here 
are I never read poetry ' m not good with poems or 
I love the beach — it helps me put my mind off things 



PR ^ Perton^ii— analytic. Respondent gives 
personal reactions to content in an analytic sense — 
identification with characters, judgments about actions 
of characters and advice giving, observations about the 
way society should/does work. Respondent states, for 
example "I might have felt the same thing." "It describes 
my feelings of moving to a new state," or "I feel that the 
poem IS right by talking about the real problems of air 
pollution facing us " 

X= Personal — global. Respondent gives personal 
reactions to genre and content in a global sense 
Examples would be statements of the following type "I 
like poems about nature." "I can relate to this poem." 
"f wish I could write poems like this." or This is not my 
kind of poem." 

EM ^ Emotional. Respondent attributes emotions or 
feelings of mood to the text or makes a direct statement of 
emotion. Examples would include: The poem was sad." 
"It's touching." "It had a funny feeling." "It was very 
dramatic," "It gave a happy point of view," or The ending 
makes you feel sorry for him." 

RT = Retelling. Respondent summarizes or retells 
the poem or part(s) of it. This can include statements 
referencing speciffc words or lines. (Disregard 
inaccuracies.) 

IN = inferencing. Respohdent goes beyond the text 
and provides motivations for characters or develops 
action. It includes text- based hypotheses of what did 
happen or predictions about what will happen, for 
example' "The author is longing for the home he once 
had," or "A E Housman seems to be talking about a 
country that h . been badly damaged or destroyed 

QN = Generalization. Respondent derives general 
meanings frorr the poem, such as Inside a man s heart 
live his fondest memories ' 

AN = Analyait'-auperficiai. Respondent mentions 
superficial characteristics of the text This includes 
concerns about format for example The poem doesn t 
rhyme," The poem seems more like a story It doesn t 
give the place and time,' The author uses imaginative 
language, or There is a sense of lost beauty m the 
poem " 

Y ^ Analysis— elaborated. Respondent gives an 
eial>orated or substantive discussion any one of the 
following special features or literary devices plot 
characters, setting, images, sounds, and so on Included 
here are discussions of plot veracity and meaningfuiness 
such as Even though Housman wrote this poem m 1890 
It IS still pertinent and neantngful today or Each of us 
has memories of places and people we would like to 
relive, but which time will not allow us to 

OW 0:herworl(S— general. Respondent classifies 
the work as to genre or type and compares the poem to 
other types of works or art forms in general, such as It s 
not like a poem I ve seen before or It is (ike a myth 

Z Other worlcs-- specific. Respondent compares 
the poem to a specific work which is mentioned by title, 
such as The Bible describes heaven this way 
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EV = Evaluation. Respondent judges the worth of 
the work This also includes such statements as: "It is 
stupid." "I didn't like it," "It doesn't make sense," "It is 
nicely written," or "It is imaginative." 

NOTE. In addition to the papers which were 
considered ratable (1 = ratable) and which were analyzed 
using the categories described above, some papers were 
not considered ratable and these were placed in one of 
the following classifications. 

0 ^ No response. 

2 = Ncnratabie. Copies or circular 

7 = Illegible, illiterate. 

8 = Totally off*task. 

9 = "I don't know." 



11. As The Cat Exercise 

Here is a poem about which you i.re going to be asked 
two questions I will read the poem to you as you road 
It to yourself When we have read the poem carefully, i ^iil 
read you the first question and you are fill in the oval beside 
the answer you think best Then I will read the second 
question lo you and you are to write your answer in the 
answer space 



As the cat 
climbed over 
the top of 

the jamcloset 
first the right 
forefoot 

carefully 
then the hind 
stepped down 

>nto the pit of 
the empty 
flowerpot 

Wilham Carlos Wiitiams | 



Source Information: Poem from William Carlos 
Williams COLLECTED EARLIER POEMS Copyright < 
1938 by New Directions Publishing Corporation Reprinted 
by permission of New Directions 

A Which Qf the following do you think the poet is 
really domg'? 

n He IS worried about the cat 

' . He IS bemg mad at the cat 

Li He IS bemg sad about the cat 

■ Heisdesc Jing the cat s movements 

n I dont know 

B What are your reasons for choosing your answer to 
question A"? 



Scoring Guide --Responding 
to Literature 

Explaining Responses to Literature 

Inferencing 

"As the Cat" 

4-'»O2002-22B-1 

Age 9, Package 8, Exercise 11 

General Scoring Rationale: Since the responding to 
literature objective was formulated lo address * deliberate, 
conscious kinds of interpretation." a successful response 
not only should identify the nature, meaning or purpose of 
the poem but also should explain ;,ie given inlerprelalion 
by relating it to the text. Evidence can be given by citing 
specific events in the text or special aspects of the 
construction of the text 

Scoring Guide Categories: 

i. First Categorization —Substantiation of choice of 
foil in mjltiple choice part of exercise 

2 - Ctioiceis unsubstantiated. Responses tend (o 
provide 1) circular evidence, such as It was boring"; 2) 
copying of the text with or without minor inaccuracies, 3) 
vague reasons such as The way it sounds," "The way the 
author wrote it." ' The words used," "The way it is made, ' 

' Because he show the movements," "He was worried 
about the cat," "Because he was talking about the cat's 
movements," "It sounds like the cat s movements." "It 
sounds like what happened in the poem, " and so on, or 
4) only a subjective reaction as substantiation 

3 ^ Clioice substantiated witii minimai evidence. 
Respondents substantiate their choice(s) with oniy one 
reason or piece of evidence related to the text 

A. Reason can be directly related to the text, for 
example ' Because he told the cat's every movement", 
"Because he show the movement ", "Telling how the cat 
movements was ", He is telling what the cat's movements 
are"; "Because he did something". Thai's because the 
cat IS doing"; or Because he has the words jamcloset. 
forefoot, hind " 

B. Reason can be inferred from the text, for example 
Because the cat jumped around " 

C. Reas'^'^ can be inaccurate, if it is related to the text, 
for example, worried because the cat might fall ' or 

' Because the cat was lost ' 

0. Reason can be based on personal experience that 
IS related to the text, such as "He worried because your 
mom wornes about you when you are not there 

E. Reason can refer to (bu! not retell) specific places 
in the text, such as Because of what it says in the 
last paragraph ' 
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4 ^ Cholc« subttantlattd. Respondents substantiate 
their choice(s) with at Itatt two reasons or pieces of 
evidence related to the text However, the evidence may 
be presented in an ambiguous fashion or be of the types 
described m 3C-3E, such as ^Because he is teihng how 
he IS moving and how fast. " Wilham is telling how he is 
jumping and running,' "First the cat put the right forefoot 
out and then carefully the hind stepped down/' or 'The 
poet IS describing the cat's movement in how the cat is 
doing." Reasons must be distinct— not instances of the 
same reason It may take, for example, two bits of 
information to make a single point, such as 'He describes 
the cat's movements when he is going in the flowerpot. ' 
"Forefoot and then the hind steps down, ' or " He is 
describing the cat's movements What he is doing " 

5 ^ Choict substantlatad In a coherent fashion. 
Respondents substantiate their choice(s) with at laatt 
two reasons or pieces of evidence related to thd text 
In addition, the reasons are presented logically and 
coherently, and are claarty related to the text, such as It 
tells at! about how he got down from the jamcloset and 
landed m the flowerpot ' 

NOTE. The following types of papers were classified as 
indicated and received no further scoring 

0 = No raaponsa. 

7 - lllaglbla or llllterata. 

8 - Totally pff-task. 

9 ^ "I don*t know." 

NOTE There is no score point 1 for this exercise 

II. S«€ond Categorization — The source of the 
evidence Code presence or absence for each of the 
following 

1 ^ Content. The evidence is basec^ on the content m 
the text 

2 ^ Form. The evidence is based on the language, 
style or construction of the text 

3 ^ Subjective reactions. Some responses judge 
the worth of ail or part of the text, for example i dislike 
poetry ' or "It was interesting " Personal opinions about the 
actions of the characters are stated or references to the 
moral of the story or general philosophical statements 
"Keep trying " 

NOTE Content and form can be present only if primary 
categorization is a 3' through 5 subjective reactions 
can be present in papers categorized 2' {hrough 5 

III. Third Categorization — A count of the number of 
reasons or pieces of evidence Categonzation for the count 
of details IS as follows (1).(2).(3 ,(4). (5). (6) (7 or more) 
NOTE This count only applies to papers with primary 
categorization of 3 through 5 ', subjective reactions 
should not be counted as reasons or evidence 



The Closing of the Rodeo 

The lariat snaps, the cowboy rolls 
His pack, and mounts and rides away 
Back to the land the cowboy goes 

Plumes of smoke from the factory sway 
In the setting sun The curtain falls, 
A train in the darkness pulls away 

Goodbye, says the ram on the iron roofs 

Goodbye, say the barber poles 

Dark drum the vanishing horses' hooves 



William Jay Smith 



Source Information: The Closing of the Rodeo ' 
Reprinted from NEW AND SELECTED POEMS by Wil'iam 
Jay Smith Copyright (, 1947. 1970 by William Jay Smith 
and used by permission of the publisher. Oelacorte 
press/Seymour Lawrence 

A. Which of the following do you think describes the 
mood of this poem*^ 

Angry 
1. Cheerful 
r Humorous 
■ Sad 

Zl I don't know 

B. What are your reasons for choosing your answer 
to Question A*? 



Scoring Guide — Responding 
to Literature 
Explaining Responses 
to Literature 

Inferencing — Mood 
'Rodeo** 

4-202024-22B-2, 3 

Age 13, Package 7, Exercise 11 

Age 17, Package 7, Exercise 9 



12. Rodeo Exercise 

Here is a poem about which you are gomg to be asked 
two questions I will read the poem aloud as you read it 
to yourself When we have read the poem carefully t will 
read you the first question and you are to fill m the box 
beside the answer you think best Then i will read the 
second question to you and you are to write your answer 
in the answer space 



General Scoring Rationale: Since the responding to 
literature objective was formulated to address deliberate 
conscious kinds of interpretation a successful response 
not only should identify the mood but also should explain 
the given interpretation by relating it to the text Evidence 
can be given by citmg specific events m the text or special 
aspects of the construction of the text 
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Seorlno Quidi Cat«QOfits: 

I. First CGt«ooriiatlon -^Substantiation of mood. 

2 « Mood Is unsubttantlatod. Respondents do not 
substtntltte their choice(s) with evidence from the text. 
Responses tend to provide: 1) circular evidence such 
as: "It was sad because it was sad"; 2) a copy or close 
paraphra^ of the text; 3) vague reasons like "The way it 
sounds," "The way the author wrote it," "The words 
usod," "The way it is made," and so on; or 4) only a 
subjective reaction as substantiation 

3 = Mood substentletod with minlmsl svldsnce. 
Respondents substantiate their choice(s) of mood with 
only ono reason or piece of evidence related to the text. 

A. Reason can be directly related to the text, such 
as* "It was sad, because he was all alone " 

B. Reason can be inferred from the text, such as: 
"Sad. because the cowboy lost the rodeo " 

C. Reason can be inaccurate, if it is related to the 
text, such as. Humorous, because the cowboy rolls" or 

Sad. because of pollution." 

0. Reason can be based on personal experience 
that IS related to the text 

E. Reason can refer to (but not retell) specific 
places in the text, such as. "Because of what it says in the 
test paragraph " NOTE. The way he read it" is not a 
reason, since ft is not related to the text 

4 = Mood substantlstsd. Respondents substantiate 
their choice(s) of mood with st least two reasons or 
pieces of evidence related to the text However, the 
evidence may be presented in an ambiguous fashion or 
be of the types described in 3C-3E Reasons must be 
distinct —not instances of the same reason It may take, 
for example, two bits of information to make a single point 
such as (Sad) because he liked to be in the rodeo, but 

It was over 



5 = Mood substantlsted In s cohsrsnt fsshlon. 

Respondents substantiate their choice(s) of moodwith at 
least two reasons or pieces of evidence cissriy related to 
the text—directly related, or can be readily inferred. The 
rewo'^s are presented logically and coherently 

NOT E: The following types of papers were classified 
as indicated and received no further scoring. 

0 ^ No rosponss. 

7^ lllsglbieor lllltsrsts. 

Totally off-tssk. 
9= "I don't know." 

NOTE: There is no score point 1 for this exercise. 

II. Second Categorization— The source of the 
evidence. Code presence or absence for each of the 
following: 

1 = Content. The evidence ts based on the content in 
the text, for example: "Everything got dark." 

2 = Form- The evidence is based on the language, 
style or construction of the text, for example* it has slow, 
low talk." 

3 Subjoctlvs reactions. These are responses 
that judge the worth of all or part of the text, for example. 
"I dislike poetry," "It was interesting," or "It was 
monotortous." Personal opinions about the actions of the 
charaotiBrs are stated or references to the moral of the 
story or general philosophical statements, such as: "Keep 
trying." 

NOTE: Content or form can be present only if the 
primary categorization is a "3" through "5". subjective 
reactions can be present in papers categorized "2" 
through "5/' 

III. Third Categorization— A count of the number 
of reasons or pieces of evidence Categorization for the 
count of details is as follows: (1), (2). (3), (4). (5). (6), (7 
or more). NOTE: This count only applies to papers with 
primary categorization of "3" through "5"; Subjective 
reactions should not be counted as reasons or evidence. 
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READING 

Twelve reports from the first assessment of reading m 1970-71 
(Write to the address below for titles and prices) 

1974-75 Assessment 

06-R'01 Reading m America A Perspective on Two Assessments, October 1976 1 40 

06-R-21 Reading Change, 1970-75 Summary Volume, April 1978 2 75 

ADULTS (special probe) 
1976-77 Assessment • 

08-YA-25 Technical Information and Data From the 1977 Ytung Adult Assessment 

of Health, Energy and Reading, March 1979 15 QO 

08-R-51 Adult Readers Will They Need Basics Too'> October 1979 1 00 

LITERATURE 

Six reports from the first assessment of literature in 1970-71 
(Write to the address below for titles and prices) 

READING/ LITERATURE 
1979-80 Asse^'sment 

11-R-01 $hree National Assessments of Reading Changes m 

Performance. 1970-80, April 1981 2 65 
11-L-01 Reading. Thinking and Writing Results From the 1979-80 

Assessment of Reading and Literature, October 1981 9 80 

11-RL-25 Reading/ Literature Released Exercise Set, April 1981 30 

11-RL-26 Supp/emenf fo 7 7-W/.-25 (sample responses) 33 00 

11-RL-40 Procedural Handbook Reading and Literature Assessment, September 1981 7 90 

In addition to the above reports. National Assessment has produced reports m the areas of social studies. 
Citizenship, music, art. mathematics, science and career and occupational development A complete 
publications list and ordering mforrriation are available from the address below 



NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS 
Su;te 700, 1860 Lincoln Street 
Denver, Colorado 80295 



wo 



ERLC 



